[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971406251336-25971-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 23:10:06 dist-test-slave-ss4n krb5kdc[25978](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 23:10:06 dist-test-slave-ss4n krb5kdc[25978](info): set up 2 sockets
Aug 05 23:10:06 dist-test-slave-ss4n krb5kdc[25978](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260805 23:10:07.486235 25971 mini_kdc.cc:121] Time spent starting KDC: real 1.167s user 0.001s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971406251336-25971-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971406251336-25971-0/krb5kdc/test-user.keytab.
Aug 05 23:10:07 dist-test-slave-ss4n krb5kdc[25978](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971407, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T23:10:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:10:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.92.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.92.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.92.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab.
Entry for principal kudu/127.25.92.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab.
I20260805 23:10:07.637360 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971406251336-25971-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971406251336-25971-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971406251336-25971-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.254:34337
--webserver_interface=127.25.92.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab
--principal=kudu/127.25.92.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:42781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.92.254:34337
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:10:07.780683 25991 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:10:07.780997 25991 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:10:07.781040 25991 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:10:07.785346 25991 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:10:07.785418 25991 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:10:07.785437 25991 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:10:07.785454 25991 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:10:07.785470 25991 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:10:07.791733 25991 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:42781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971406251336-25971-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971406251336-25971-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.92.254:34337
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab
--principal=kudu/127.25.92.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.254:34337
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.92.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25991
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971406251336-25971-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:10:07.793382 25991 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:10:07.794780 25991 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:10:07.807799 25997 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:10:07.808036 25991 server_base.cc:1034] running on GCE node
W20260805 23:10:07.808702 26000 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:07.809032 25998 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:10:07.809952 25991 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:10:07.811810 25991 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:10:07.813045 25991 hybrid_clock.cc:648] HybridClock initialized: now 1785971407813025 us; error 42 us; skew 500 ppm
Aug 05 23:10:07 dist-test-slave-ss4n krb5kdc[25978](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971407, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:10:07.816443 25991 init.cc:377] Logged in from keytab as kudu/127.25.92.254@KRBTEST.COM (short username kudu)
I20260805 23:10:07.817826 25991 webserver.cc:460] Webserver started at http://127.25.92.254:42621/ using document root <none> and password file <none>
I20260805 23:10:07.818773 25991 fs_manager.cc:359] Metadata directory not provided
I20260805 23:10:07.818881 25991 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:10:07.819305 25991 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:10:07.821887 25991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971406251336-25971-0/minicluster-data/master-0/data/instance:
uuid: "8ed2d4cf907a4781bd07e04662ed5068"
format_stamp: "Formatted at 2026-08-05 23:10:07 on dist-test-slave-ss4n"
server_key: "f13cbd7a049ecd3d3fff51e3bb66a30f"
server_key_iv: "f9277f67f4e0b238b7c4bb107819a7c5"
server_key_version: "encryptionkey@0"
I20260805 23:10:07.822497 25991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971406251336-25971-0/minicluster-data/master-0/wal/instance:
uuid: "8ed2d4cf907a4781bd07e04662ed5068"
format_stamp: "Formatted at 2026-08-05 23:10:07 on dist-test-slave-ss4n"
server_key: "f13cbd7a049ecd3d3fff51e3bb66a30f"
server_key_iv: "f9277f67f4e0b238b7c4bb107819a7c5"
server_key_version: "encryptionkey@0"
I20260805 23:10:07.827580 25991 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260805 23:10:07.831604 26007 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:07.833689 25991 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260805 23:10:07.833945 25991 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971406251336-25971-0/minicluster-data/master-0/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971406251336-25971-0/minicluster-data/master-0/wal
uuid: "8ed2d4cf907a4781bd07e04662ed5068"
format_stamp: "Formatted at 2026-08-05 23:10:07 on dist-test-slave-ss4n"
server_key: "f13cbd7a049ecd3d3fff51e3bb66a30f"
server_key_iv: "f9277f67f4e0b238b7c4bb107819a7c5"
server_key_version: "encryptionkey@0"
I20260805 23:10:07.834115 25991 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971406251336-25971-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971406251336-25971-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971406251336-25971-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:10:07.858219 25991 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:10:07.862592 25991 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:10:07.862835 25991 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:10:07.873450 25991 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.254:34337
I20260805 23:10:07.873663 26062 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.254:34337 every 8 connection(s)
I20260805 23:10:07.875286 25991 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
I20260805 23:10:07.876613 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 25991
I20260805 23:10:07.876742 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971406251336-25971-0/minicluster-data/master-0/wal/instance
I20260805 23:10:07.878280 25971 external_mini_cluster.cc:1428] Setting key db1697502eb4e71715d57bc9914c8925
I20260805 23:10:07.880606 26063 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:10:07.891295 26063 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ed2d4cf907a4781bd07e04662ed5068: Bootstrap starting.
I20260805 23:10:07.895289 26063 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8ed2d4cf907a4781bd07e04662ed5068: Neither blocks nor log segments found. Creating new log.
I20260805 23:10:07.896338 26063 log.cc:824] T 00000000000000000000000000000000 P 8ed2d4cf907a4781bd07e04662ed5068: Log is configured to *not* fsync() on all Append() calls
Aug 05 23:10:07 dist-test-slave-ss4n krb5kdc[25978](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971407, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM
I20260805 23:10:07.899255 26063 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ed2d4cf907a4781bd07e04662ed5068: No bootstrap required, opened a new log
I20260805 23:10:07.904814 26065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:07.882978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44228 (local address 127.25.92.254:34337)
0805 23:10:07.883521 (+ 543us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:07.883539 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:07.884946 (+ 1407us) server_negotiation.cc:407] Connection header received
0805 23:10:07.888393 (+ 3447us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:07.888409 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:07.888825 (+ 416us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:07.889167 (+ 342us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:07.891795 (+ 2628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:07.892858 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:07.894179 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:07.894648 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:07.894836 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:07.899025 (+ 4189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:07.899071 (+ 46us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:07.899083 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:07.899114 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:07.900952 (+ 1838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:07.901589 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:07.901594 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:07.901599 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:07.901716 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:07.902092 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:07.902096 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:07.902097 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:07.902561 (+ 464us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:07.902798 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:07.903413 (+ 615us) server_negotiation.cc:299] Negotiation successful
0805 23:10:07.903733 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":193,"threads_started":1}
I20260805 23:10:07.904747 26063 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8ed2d4cf907a4781bd07e04662ed5068 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ed2d4cf907a4781bd07e04662ed5068" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 34337 } }
I20260805 23:10:07.905179 26063 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8ed2d4cf907a4781bd07e04662ed5068 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:10:07.905299 26063 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8ed2d4cf907a4781bd07e04662ed5068 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ed2d4cf907a4781bd07e04662ed5068, State: Initialized, Role: FOLLOWER
I20260805 23:10:07.906383 26063 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8ed2d4cf907a4781bd07e04662ed5068 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ed2d4cf907a4781bd07e04662ed5068" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 34337 } }
I20260805 23:10:07.906710 26063 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8ed2d4cf907a4781bd07e04662ed5068 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:10:07.906860 26063 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8ed2d4cf907a4781bd07e04662ed5068 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:10:07.907022 26063 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8ed2d4cf907a4781bd07e04662ed5068 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:10:07.908504 26063 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8ed2d4cf907a4781bd07e04662ed5068 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ed2d4cf907a4781bd07e04662ed5068" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 34337 } }
I20260805 23:10:07.909551 26063 leader_election.cc:302] T 00000000000000000000000000000000 P 8ed2d4cf907a4781bd07e04662ed5068 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8ed2d4cf907a4781bd07e04662ed5068; no voters:
I20260805 23:10:07.910040 26063 leader_election.cc:288] T 00000000000000000000000000000000 P 8ed2d4cf907a4781bd07e04662ed5068 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 23:10:07.910220 26068 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8ed2d4cf907a4781bd07e04662ed5068 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:10:07.910548 26068 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8ed2d4cf907a4781bd07e04662ed5068 [term 1 LEADER]: Becoming Leader. State: Replica: 8ed2d4cf907a4781bd07e04662ed5068, State: Running, Role: LEADER
I20260805 23:10:07.910974 26068 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8ed2d4cf907a4781bd07e04662ed5068 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ed2d4cf907a4781bd07e04662ed5068" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 34337 } }
I20260805 23:10:07.911603 26063 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8ed2d4cf907a4781bd07e04662ed5068 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:10:07.913777 26070 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ed2d4cf907a4781bd07e04662ed5068 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8ed2d4cf907a4781bd07e04662ed5068. Latest consensus state: current_term: 1 leader_uuid: "8ed2d4cf907a4781bd07e04662ed5068" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ed2d4cf907a4781bd07e04662ed5068" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 34337 } } }
I20260805 23:10:07.913899 26069 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ed2d4cf907a4781bd07e04662ed5068 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8ed2d4cf907a4781bd07e04662ed5068" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ed2d4cf907a4781bd07e04662ed5068" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 34337 } } }
I20260805 23:10:07.914060 26069 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ed2d4cf907a4781bd07e04662ed5068 [sys.catalog]: This master's current role is: LEADER
I20260805 23:10:07.913971 26070 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ed2d4cf907a4781bd07e04662ed5068 [sys.catalog]: This master's current role is: LEADER
I20260805 23:10:07.914733 26077 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:10:07.920811 26077 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:10:07.931557 26077 catalog_manager.cc:1347] Generated new cluster ID: dbd75984fd864737aa4d1ca853968dcb
I20260805 23:10:07.931658 26077 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:10:07.955885 26077 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:10:07.957175 26077 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:10:07.965948 26077 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8ed2d4cf907a4781bd07e04662ed5068: Generated new TSK 0
I20260805 23:10:07.967034 26077 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.92.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.92.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.92.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.193.keytab.
Entry for principal kudu/127.25.92.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.193.keytab.
I20260805 23:10:08.013578 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971406251336-25971-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971406251336-25971-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971406251336-25971-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971406251336-25971-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.92.193:0
--local_ip_for_outbound_sockets=127.25.92.193
--webserver_interface=127.25.92.193
--webserver_port=0
--tserver_master_addrs=127.25.92.254:34337
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.193.keytab
--principal=kudu/127.25.92.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:42781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:10:08.147949 26090 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:10:08.148306 26090 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:10:08.148362 26090 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:10:08.152861 26090 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:10:08.152952 26090 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:10:08.153020 26090 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.92.193
I20260805 23:10:08.158843 26090 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:42781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971406251336-25971-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971406251336-25971-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.193.keytab
--principal=kudu/127.25.92.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.92.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971406251336-25971-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.92.193
--webserver_port=0
--tserver_master_addrs=127.25.92.254:34337
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26090
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.92.193
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971406251336-25971-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:10:08.160148 26090 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:10:08.161299 26090 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:10:08.172856 26098 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:08.172986 26095 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:08.173007 26096 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:10:08.173296 26090 server_base.cc:1034] running on GCE node
I20260805 23:10:08.173830 26090 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:10:08.174618 26090 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:10:08.175856 26090 hybrid_clock.cc:648] HybridClock initialized: now 1785971408175817 us; error 53 us; skew 500 ppm
Aug 05 23:10:08 dist-test-slave-ss4n krb5kdc[25978](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971408, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:10:08.179318 26090 init.cc:377] Logged in from keytab as kudu/127.25.92.193@KRBTEST.COM (short username kudu)
I20260805 23:10:08.180536 26090 webserver.cc:460] Webserver started at http://127.25.92.193:38017/ using document root <none> and password file <none>
I20260805 23:10:08.181152 26090 fs_manager.cc:359] Metadata directory not provided
I20260805 23:10:08.181247 26090 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:10:08.181546 26090 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:10:08.183772 26090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971406251336-25971-0/minicluster-data/ts-0/data/instance:
uuid: "14f9022fd8ac494b96a15ce5eedcec28"
format_stamp: "Formatted at 2026-08-05 23:10:08 on dist-test-slave-ss4n"
server_key: "7fda079db3dde56e6a7de4478be4f881"
server_key_iv: "8205dd9961d9e96c9e8778c5dc4c02f3"
server_key_version: "encryptionkey@0"
I20260805 23:10:08.184384 26090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971406251336-25971-0/minicluster-data/ts-0/wal/instance:
uuid: "14f9022fd8ac494b96a15ce5eedcec28"
format_stamp: "Formatted at 2026-08-05 23:10:08 on dist-test-slave-ss4n"
server_key: "7fda079db3dde56e6a7de4478be4f881"
server_key_iv: "8205dd9961d9e96c9e8778c5dc4c02f3"
server_key_version: "encryptionkey@0"
I20260805 23:10:08.188424 26090 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260805 23:10:08.191565 26105 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:08.192992 26090 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260805 23:10:08.193168 26090 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971406251336-25971-0/minicluster-data/ts-0/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971406251336-25971-0/minicluster-data/ts-0/wal
uuid: "14f9022fd8ac494b96a15ce5eedcec28"
format_stamp: "Formatted at 2026-08-05 23:10:08 on dist-test-slave-ss4n"
server_key: "7fda079db3dde56e6a7de4478be4f881"
server_key_iv: "8205dd9961d9e96c9e8778c5dc4c02f3"
server_key_version: "encryptionkey@0"
I20260805 23:10:08.193321 26090 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971406251336-25971-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971406251336-25971-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971406251336-25971-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:10:08.205984 26090 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:10:08.209992 26090 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:10:08.210263 26090 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:10:08.211105 26090 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:10:08.212476 26090 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:10:08.212566 26090 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:08.212636 26090 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:10:08.212698 26090 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:08.225006 26090 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.193:37273
I20260805 23:10:08.225121 26220 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.193:37273 every 8 connection(s)
I20260805 23:10:08.226153 26090 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971406251336-25971-0/minicluster-data/ts-0/data/info.pb
I20260805 23:10:08.232234 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 26090
I20260805 23:10:08.232466 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971406251336-25971-0/minicluster-data/ts-0/wal/instance
I20260805 23:10:08.232723 25971 external_mini_cluster.cc:1428] Setting key 55f02db799f7cf444057ce6da1ced2ab
Aug 05 23:10:08 dist-test-slave-ss4n krb5kdc[25978](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971408, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.193@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.25.92.194@KRBTEST.COM; defaulting to no policy
I20260805 23:10:08.244989 26065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:08.228781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.193:41401 (local address 127.25.92.254:34337)
0805 23:10:08.228973 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:08.228977 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:08.230306 (+ 1329us) server_negotiation.cc:407] Connection header received
0805 23:10:08.231278 (+ 972us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:08.231281 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:08.231333 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:08.231435 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:08.232947 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:08.233846 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:08.235947 (+ 2101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:08.236237 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:08.236357 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:08.239884 (+ 3527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:08.239891 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:08.239896 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:08.239940 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:08.242194 (+ 2254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:08.242900 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:08.242905 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:08.242907 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:08.243020 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:08.243492 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:08.243497 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:08.243501 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:08.243803 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:08.243923 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:08.244571 (+ 648us) server_negotiation.cc:299] Negotiation successful
0805 23:10:08.244754 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260805 23:10:08.245923 26223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:08.229216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:34337 (local address 127.25.92.193:41401)
0805 23:10:08.230092 (+ 876us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:08.230142 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:08.231030 (+ 888us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:08.231639 (+ 609us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:08.231647 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:08.232107 (+ 460us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:08.232745 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:08.232762 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:08.234033 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:08.234036 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:08.235768 (+ 1732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:08.235788 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:08.236465 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:08.236472 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:08.236764 (+ 292us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:08.237653 (+ 889us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:10:08.237674 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:08.239605 (+ 1931us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:10:08.242383 (+ 2778us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:08.242390 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:08.242407 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:08.242756 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:08.243178 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:08.243181 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:08.243182 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:08.243365 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:08.243945 (+ 580us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:10:08.243952 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:10:08.244282 (+ 330us) client_negotiation.cc:769] Sending connection context
0805 23:10:08.244471 (+ 189us) client_negotiation.cc:240] Negotiation successful
0805 23:10:08.244801 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":675,"thread_start_us":123,"threads_started":1}
I20260805 23:10:08.247622 26221 heartbeater.cc:347] Connected to a master server at 127.25.92.254:34337
I20260805 23:10:08.247989 26221 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.92.194@KRBTEST.COM" created.
I20260805 23:10:08.248610 26221 heartbeater.cc:511] Master 127.25.92.254:34337 requested a full tablet report, sending...
I20260805 23:10:08.251261 26025 ts_manager.cc:194] Registered new tserver with Master: 14f9022fd8ac494b96a15ce5eedcec28 (127.25.92.193:37273)
I20260805 23:10:08.253039 26025 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.92.193@KRBTEST.COM'} at 127.25.92.193:41401
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.92.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.194.keytab.
Entry for principal kudu/127.25.92.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.194.keytab.
I20260805 23:10:08.271018 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971406251336-25971-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971406251336-25971-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971406251336-25971-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971406251336-25971-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.92.194:0
--local_ip_for_outbound_sockets=127.25.92.194
--webserver_interface=127.25.92.194
--webserver_port=0
--tserver_master_addrs=127.25.92.254:34337
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.194.keytab
--principal=kudu/127.25.92.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:42781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:10:08.398564 26226 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:10:08.398882 26226 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:10:08.398936 26226 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:10:08.403316 26226 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:10:08.403401 26226 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:10:08.403467 26226 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.92.194
I20260805 23:10:08.409380 26226 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:42781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971406251336-25971-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971406251336-25971-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.194.keytab
--principal=kudu/127.25.92.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.92.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971406251336-25971-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.92.194
--webserver_port=0
--tserver_master_addrs=127.25.92.254:34337
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26226
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.92.194
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971406251336-25971-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:10:08.410602 26226 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:10:08.411624 26226 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:10:08.422483 26232 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:08.422449 26235 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:08.422449 26231 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:10:08.422894 26226 server_base.cc:1034] running on GCE node
I20260805 23:10:08.423466 26226 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:10:08.424260 26226 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:10:08.425602 26226 hybrid_clock.cc:648] HybridClock initialized: now 1785971408425450 us; error 47 us; skew 500 ppm
Aug 05 23:10:08 dist-test-slave-ss4n krb5kdc[25978](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971408, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:10:08.428944 26226 init.cc:377] Logged in from keytab as kudu/127.25.92.194@KRBTEST.COM (short username kudu)
I20260805 23:10:08.430078 26226 webserver.cc:460] Webserver started at http://127.25.92.194:34319/ using document root <none> and password file <none>
I20260805 23:10:08.430701 26226 fs_manager.cc:359] Metadata directory not provided
I20260805 23:10:08.430797 26226 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:10:08.431013 26226 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:10:08.433202 26226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971406251336-25971-0/minicluster-data/ts-1/data/instance:
uuid: "b418af1a8fab4f7faa0f9cd4d49caf2b"
format_stamp: "Formatted at 2026-08-05 23:10:08 on dist-test-slave-ss4n"
server_key: "3b5cb5e91ae13ecdb6135b6f3e9e7e2a"
server_key_iv: "1eae1e2a1bc587da8358df7cb531dabd"
server_key_version: "encryptionkey@0"
I20260805 23:10:08.433955 26226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971406251336-25971-0/minicluster-data/ts-1/wal/instance:
uuid: "b418af1a8fab4f7faa0f9cd4d49caf2b"
format_stamp: "Formatted at 2026-08-05 23:10:08 on dist-test-slave-ss4n"
server_key: "3b5cb5e91ae13ecdb6135b6f3e9e7e2a"
server_key_iv: "1eae1e2a1bc587da8358df7cb531dabd"
server_key_version: "encryptionkey@0"
I20260805 23:10:08.439121 26226 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.000s
I20260805 23:10:08.442397 26242 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:08.443941 26226 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260805 23:10:08.444111 26226 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971406251336-25971-0/minicluster-data/ts-1/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971406251336-25971-0/minicluster-data/ts-1/wal
uuid: "b418af1a8fab4f7faa0f9cd4d49caf2b"
format_stamp: "Formatted at 2026-08-05 23:10:08 on dist-test-slave-ss4n"
server_key: "3b5cb5e91ae13ecdb6135b6f3e9e7e2a"
server_key_iv: "1eae1e2a1bc587da8358df7cb531dabd"
server_key_version: "encryptionkey@0"
I20260805 23:10:08.444237 26226 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971406251336-25971-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971406251336-25971-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971406251336-25971-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:10:08.482358 26226 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:10:08.486343 26226 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:10:08.486550 26226 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:10:08.487150 26226 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:10:08.488080 26226 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:10:08.488137 26226 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:08.488185 26226 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:10:08.488204 26226 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:08.499490 26226 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.194:34595
I20260805 23:10:08.499581 26358 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.194:34595 every 8 connection(s)
I20260805 23:10:08.500538 26226 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971406251336-25971-0/minicluster-data/ts-1/data/info.pb
I20260805 23:10:08.509949 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 26226
I20260805 23:10:08.510087 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971406251336-25971-0/minicluster-data/ts-1/wal/instance
I20260805 23:10:08.510376 25971 external_mini_cluster.cc:1428] Setting key 11769fc330cb14e79c39714514b45400
Aug 05 23:10:08 dist-test-slave-ss4n krb5kdc[25978](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971408, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.194@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM
I20260805 23:10:08.518045 26065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:08.503125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.194:55429 (local address 127.25.92.254:34337)
0805 23:10:08.503314 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:08.503319 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:08.504472 (+ 1153us) server_negotiation.cc:407] Connection header received
0805 23:10:08.505601 (+ 1129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:08.505605 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:08.505675 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:08.505798 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:08.507142 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:08.507743 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:08.509021 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:08.509270 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:08.509403 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:08.512968 (+ 3565us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:08.512973 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:08.512981 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:08.513006 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:08.515260 (+ 2254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:08.515973 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:08.515980 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:08.515985 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:08.516088 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:08.516519 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:08.516531 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:08.516536 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:08.516841 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:08.516961 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:08.517721 (+ 760us) server_negotiation.cc:299] Negotiation successful
0805 23:10:08.517873 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260805 23:10:08.519047 26361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:08.503497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:34337 (local address 127.25.92.194:55429)
0805 23:10:08.504280 (+ 783us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:08.504325 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:08.505320 (+ 995us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:08.505996 (+ 676us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:08.506004 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:08.506412 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:08.506965 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:08.506980 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:08.507896 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:08.507907 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:08.508857 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:08.508868 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:08.509466 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:08.509471 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:08.509787 (+ 316us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:08.510595 (+ 808us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:10:08.510620 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:08.512769 (+ 2149us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:10:08.515454 (+ 2685us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:08.515461 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:08.515479 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:08.515846 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:08.516228 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:08.516231 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:08.516233 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:08.516403 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:08.517000 (+ 597us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:10:08.517010 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:10:08.517351 (+ 341us) client_negotiation.cc:769] Sending connection context
0805 23:10:08.517681 (+ 330us) client_negotiation.cc:240] Negotiation successful
0805 23:10:08.518057 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":580,"thread_start_us":156,"threads_started":1}
I20260805 23:10:08.520391 26359 heartbeater.cc:347] Connected to a master server at 127.25.92.254:34337
I20260805 23:10:08.520675 26359 heartbeater.cc:464] Registering TS with master...
I20260805 23:10:08.521247 26359 heartbeater.cc:511] Master 127.25.92.254:34337 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.25.92.195@KRBTEST.COM; defaulting to no policy
I20260805 23:10:08.522538 26025 ts_manager.cc:194] Registered new tserver with Master: b418af1a8fab4f7faa0f9cd4d49caf2b (127.25.92.194:34595)
I20260805 23:10:08.523164 26025 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.92.194@KRBTEST.COM'} at 127.25.92.194:55429
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.92.195@KRBTEST.COM" created.
I20260805 23:10:09.256186 26221 heartbeater.cc:503] Master 127.25.92.254:34337 was elected leader, sending a full tablet report...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.92.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.195.keytab.
Entry for principal kudu/127.25.92.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.195.keytab.
W20260805 23:10:09.417585 25971 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.25.92.195: real 0.905s user 0.000s sys 0.005s
I20260805 23:10:09.418256 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971406251336-25971-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971406251336-25971-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971406251336-25971-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971406251336-25971-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.92.195:0
--local_ip_for_outbound_sockets=127.25.92.195
--webserver_interface=127.25.92.195
--webserver_port=0
--tserver_master_addrs=127.25.92.254:34337
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.195.keytab
--principal=kudu/127.25.92.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:42781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260805 23:10:09.525620 26359 heartbeater.cc:503] Master 127.25.92.254:34337 was elected leader, sending a full tablet report...
W20260805 23:10:09.543483 26364 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:10:09.543833 26364 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:10:09.543923 26364 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:10:09.548203 26364 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:10:09.548308 26364 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:10:09.548413 26364 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.92.195
I20260805 23:10:09.554177 26364 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:42781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971406251336-25971-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971406251336-25971-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.195.keytab
--principal=kudu/127.25.92.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.92.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971406251336-25971-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.92.195
--webserver_port=0
--tserver_master_addrs=127.25.92.254:34337
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26364
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.92.195
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971406251336-25971-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:10:09.555369 26364 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:10:09.556445 26364 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:10:09.566970 26370 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:09.567198 26372 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:09.567214 26369 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:10:09.567364 26364 server_base.cc:1034] running on GCE node
I20260805 23:10:09.567898 26364 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:10:09.568650 26364 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:10:09.569864 26364 hybrid_clock.cc:648] HybridClock initialized: now 1785971409569839 us; error 42 us; skew 500 ppm
Aug 05 23:10:09 dist-test-slave-ss4n krb5kdc[25978](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971409, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:10:09.573302 26364 init.cc:377] Logged in from keytab as kudu/127.25.92.195@KRBTEST.COM (short username kudu)
I20260805 23:10:09.574445 26364 webserver.cc:460] Webserver started at http://127.25.92.195:45745/ using document root <none> and password file <none>
I20260805 23:10:09.575099 26364 fs_manager.cc:359] Metadata directory not provided
I20260805 23:10:09.575212 26364 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:10:09.575475 26364 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:10:09.577829 26364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971406251336-25971-0/minicluster-data/ts-2/data/instance:
uuid: "91744dca42214ef3b61ed2ce9a3f6943"
format_stamp: "Formatted at 2026-08-05 23:10:09 on dist-test-slave-ss4n"
server_key: "4a0897ab45f09b3dbe6964e91130d71a"
server_key_iv: "35ce13b46511881d2986bcd5342d8c5b"
server_key_version: "encryptionkey@0"
I20260805 23:10:09.578454 26364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971406251336-25971-0/minicluster-data/ts-2/wal/instance:
uuid: "91744dca42214ef3b61ed2ce9a3f6943"
format_stamp: "Formatted at 2026-08-05 23:10:09 on dist-test-slave-ss4n"
server_key: "4a0897ab45f09b3dbe6964e91130d71a"
server_key_iv: "35ce13b46511881d2986bcd5342d8c5b"
server_key_version: "encryptionkey@0"
I20260805 23:10:09.582675 26364 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260805 23:10:09.585443 26379 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:09.586956 26364 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260805 23:10:09.587134 26364 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971406251336-25971-0/minicluster-data/ts-2/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971406251336-25971-0/minicluster-data/ts-2/wal
uuid: "91744dca42214ef3b61ed2ce9a3f6943"
format_stamp: "Formatted at 2026-08-05 23:10:09 on dist-test-slave-ss4n"
server_key: "4a0897ab45f09b3dbe6964e91130d71a"
server_key_iv: "35ce13b46511881d2986bcd5342d8c5b"
server_key_version: "encryptionkey@0"
I20260805 23:10:09.587276 26364 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971406251336-25971-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971406251336-25971-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971406251336-25971-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:10:09.600108 26364 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:10:09.603703 26364 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:10:09.603938 26364 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:10:09.604646 26364 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:10:09.605630 26364 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:10:09.605703 26364 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:09.605774 26364 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:10:09.605834 26364 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:09.616405 26364 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.195:38141
I20260805 23:10:09.616425 26494 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.195:38141 every 8 connection(s)
I20260805 23:10:09.617408 26364 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971406251336-25971-0/minicluster-data/ts-2/data/info.pb
I20260805 23:10:09.625790 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 26364
I20260805 23:10:09.625931 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971406251336-25971-0/minicluster-data/ts-2/wal/instance
I20260805 23:10:09.626261 25971 external_mini_cluster.cc:1428] Setting key 6022bd816fdab11794434ec33b1afd30
Aug 05 23:10:09 dist-test-slave-ss4n krb5kdc[25978](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971409, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.195@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM
I20260805 23:10:09.633682 26497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:09.620330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.195:60993 (local address 127.25.92.254:34337)
0805 23:10:09.620641 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:09.620646 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:09.621761 (+ 1115us) server_negotiation.cc:407] Connection header received
0805 23:10:09.622696 (+ 935us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:09.622699 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:09.622762 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:09.622878 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:09.624090 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:09.624681 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:09.626174 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:09.626486 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:09.626593 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:09.629701 (+ 3108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:09.629704 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:09.629706 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:09.629732 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:09.631194 (+ 1462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:09.631833 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:09.631836 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:09.631838 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:09.631931 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:09.632357 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:09.632365 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:09.632366 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:09.632571 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:09.632694 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:09.633296 (+ 602us) server_negotiation.cc:299] Negotiation successful
0805 23:10:09.633446 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":216,"thread_start_us":106,"threads_started":1}
I20260805 23:10:09.634559 26498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:09.620794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:34337 (local address 127.25.92.195:60993)
0805 23:10:09.621607 (+ 813us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:09.621653 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:09.622480 (+ 827us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:09.623054 (+ 574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:09.623060 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:09.623442 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:09.623939 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:09.623953 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:09.624841 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:09.624844 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:09.625967 (+ 1123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:09.625981 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:09.626665 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:09.626670 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:09.626873 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:09.627730 (+ 857us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:10:09.627758 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:09.629490 (+ 1732us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:10:09.631350 (+ 1860us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:09.631356 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:09.631370 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:09.631699 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:09.632062 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:09.632066 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:09.632068 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:09.632243 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:09.632736 (+ 493us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:10:09.632744 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:10:09.633020 (+ 276us) client_negotiation.cc:769] Sending connection context
0805 23:10:09.633241 (+ 221us) client_negotiation.cc:240] Negotiation successful
0805 23:10:09.633654 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":140,"threads_started":1}
I20260805 23:10:09.635921 26495 heartbeater.cc:347] Connected to a master server at 127.25.92.254:34337
I20260805 23:10:09.636277 26495 heartbeater.cc:464] Registering TS with master...
I20260805 23:10:09.636855 26495 heartbeater.cc:511] Master 127.25.92.254:34337 requested a full tablet report, sending...
I20260805 23:10:09.638235 26025 ts_manager.cc:194] Registered new tserver with Master: 91744dca42214ef3b61ed2ce9a3f6943 (127.25.92.195:38141)
I20260805 23:10:09.638907 26025 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.92.195@KRBTEST.COM'} at 127.25.92.195:60993
I20260805 23:10:09.641620 25971 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 05 23:10:09 dist-test-slave-ss4n krb5kdc[25978](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971409, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 05 23:10:09 dist-test-slave-ss4n krb5kdc[25978](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971409, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.25.92.193@KRBTEST.COM
I20260805 23:10:09.735314 26503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:09.723805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42234 (local address 127.25.92.193:37273)
0805 23:10:09.724204 (+ 399us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:09.724214 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:09.724247 (+ 33us) server_negotiation.cc:407] Connection header received
0805 23:10:09.724315 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:09.724325 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:09.724591 (+ 266us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:09.724760 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:09.725747 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:09.726715 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:09.727782 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:09.728245 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:09.728442 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:09.731353 (+ 2911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:09.731365 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:09.731381 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:09.731411 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:09.733242 (+ 1831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:09.733814 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:09.733820 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:09.733821 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:09.733904 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:09.734284 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:09.734287 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:09.734288 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:09.734552 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:09.734763 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:09.734929 (+ 166us) server_negotiation.cc:299] Negotiation successful
0805 23:10:09.735100 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":98,"threads_started":1}
W20260805 23:10:09.736299 26134 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42234
Aug 05 23:10:09 dist-test-slave-ss4n krb5kdc[25978](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971409, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 05 23:10:09 dist-test-slave-ss4n krb5kdc[25978](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971409, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.92.193@KRBTEST.COM
I20260805 23:10:09.841149 26503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:09.825109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42244 (local address 127.25.92.193:37273)
0805 23:10:09.825275 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:09.825282 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:09.825596 (+ 314us) server_negotiation.cc:407] Connection header received
0805 23:10:09.825707 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:09.825710 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:09.825760 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:09.825850 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:09.826944 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:09.827499 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:09.828419 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:09.828650 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:09.828751 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:09.836518 (+ 7767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:09.836522 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:09.836524 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:09.836562 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:09.839075 (+ 2513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:09.839661 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:09.839664 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:09.839665 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:09.839756 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:09.840150 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:09.840154 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:09.840155 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:09.840352 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:09.840514 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:09.840889 (+ 375us) server_negotiation.cc:299] Negotiation successful
0805 23:10:09.841013 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260805 23:10:09.842921 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 26090
I20260805 23:10:09.848814 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 26226
I20260805 23:10:09.855002 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 26364
I20260805 23:10:09.862370 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 25991
2026-08-05T23:10:09Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (3580 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971406251336-25971-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 23:10:10 dist-test-slave-ss4n krb5kdc[26510](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 23:10:10 dist-test-slave-ss4n krb5kdc[26510](info): set up 2 sockets
Aug 05 23:10:10 dist-test-slave-ss4n krb5kdc[26510](info): commencing operation
krb5kdc: starting...
W20260805 23:10:11.068086 25971 mini_kdc.cc:121] Time spent starting KDC: real 1.181s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971406251336-25971-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971406251336-25971-0/krb5kdc/test-user.keytab.
Aug 05 23:10:11 dist-test-slave-ss4n krb5kdc[26510](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971411, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T23:10:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:10:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.92.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.92.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.92.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab.
Entry for principal kudu/127.25.92.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab.
I20260805 23:10:11.205173 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971406251336-25971-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971406251336-25971-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971406251336-25971-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.254:46093
--webserver_interface=127.25.92.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab
--principal=kudu/127.25.92.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:41251
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.92.254:46093
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:10:11.323437 26523 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:10:11.323812 26523 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:10:11.323894 26523 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:10:11.328248 26523 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:10:11.328383 26523 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:10:11.328420 26523 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:10:11.328449 26523 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:10:11.328490 26523 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:10:11.334726 26523 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:41251
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971406251336-25971-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971406251336-25971-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.92.254:46093
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab
--principal=kudu/127.25.92.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.254:46093
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.92.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26523
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971406251336-25971-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:10:11.336014 26523 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:10:11.337256 26523 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:10:11.347441 26528 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:11.347458 26531 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:11.347427 26529 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:10:11.347927 26523 server_base.cc:1034] running on GCE node
I20260805 23:10:11.348726 26523 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:10:11.350078 26523 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:10:11.351375 26523 hybrid_clock.cc:648] HybridClock initialized: now 1785971411351327 us; error 70 us; skew 500 ppm
Aug 05 23:10:11 dist-test-slave-ss4n krb5kdc[26510](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971411, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:10:11.355490 26523 init.cc:377] Logged in from keytab as kudu/127.25.92.254@KRBTEST.COM (short username kudu)
I20260805 23:10:11.356782 26523 webserver.cc:460] Webserver started at http://127.25.92.254:40261/ using document root <none> and password file <none>
I20260805 23:10:11.357793 26523 fs_manager.cc:359] Metadata directory not provided
I20260805 23:10:11.357878 26523 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:10:11.358065 26523 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:10:11.360373 26523 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971406251336-25971-0/minicluster-data/master-0/data/instance:
uuid: "4d5f17e008314c2587eaa348c6ac8810"
format_stamp: "Formatted at 2026-08-05 23:10:11 on dist-test-slave-ss4n"
server_key: "b0064065e3d2dfef4626dc223e6e2271"
server_key_iv: "49ef71c1d898c592e81122faca752b5e"
server_key_version: "encryptionkey@0"
I20260805 23:10:11.360996 26523 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971406251336-25971-0/minicluster-data/master-0/wal/instance:
uuid: "4d5f17e008314c2587eaa348c6ac8810"
format_stamp: "Formatted at 2026-08-05 23:10:11 on dist-test-slave-ss4n"
server_key: "b0064065e3d2dfef4626dc223e6e2271"
server_key_iv: "49ef71c1d898c592e81122faca752b5e"
server_key_version: "encryptionkey@0"
I20260805 23:10:11.365410 26523 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260805 23:10:11.368304 26538 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:11.370100 26523 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:10:11.370235 26523 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971406251336-25971-0/minicluster-data/master-0/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971406251336-25971-0/minicluster-data/master-0/wal
uuid: "4d5f17e008314c2587eaa348c6ac8810"
format_stamp: "Formatted at 2026-08-05 23:10:11 on dist-test-slave-ss4n"
server_key: "b0064065e3d2dfef4626dc223e6e2271"
server_key_iv: "49ef71c1d898c592e81122faca752b5e"
server_key_version: "encryptionkey@0"
I20260805 23:10:11.370352 26523 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971406251336-25971-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971406251336-25971-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971406251336-25971-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:10:11.385154 26523 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:10:11.388881 26523 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:10:11.389057 26523 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:10:11.396733 26523 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.254:46093
I20260805 23:10:11.396763 26590 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.254:46093 every 8 connection(s)
I20260805 23:10:11.397823 26523 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
I20260805 23:10:11.401047 26591 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:10:11.403002 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 26523
I20260805 23:10:11.403131 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971406251336-25971-0/minicluster-data/master-0/wal/instance
I20260805 23:10:11.403430 25971 external_mini_cluster.cc:1428] Setting key 9a2c6a4fc9f8f5c56c0cf6081444085b
I20260805 23:10:11.408535 26591 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d5f17e008314c2587eaa348c6ac8810: Bootstrap starting.
I20260805 23:10:11.411152 26591 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4d5f17e008314c2587eaa348c6ac8810: Neither blocks nor log segments found. Creating new log.
I20260805 23:10:11.412083 26591 log.cc:824] T 00000000000000000000000000000000 P 4d5f17e008314c2587eaa348c6ac8810: Log is configured to *not* fsync() on all Append() calls
Aug 05 23:10:11 dist-test-slave-ss4n krb5kdc[26510](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971411, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM
I20260805 23:10:11.414589 26591 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d5f17e008314c2587eaa348c6ac8810: No bootstrap required, opened a new log
I20260805 23:10:11.419548 26591 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4d5f17e008314c2587eaa348c6ac8810 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d5f17e008314c2587eaa348c6ac8810" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 46093 } }
I20260805 23:10:11.419874 26591 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4d5f17e008314c2587eaa348c6ac8810 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:10:11.419963 26591 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4d5f17e008314c2587eaa348c6ac8810 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d5f17e008314c2587eaa348c6ac8810, State: Initialized, Role: FOLLOWER
I20260805 23:10:11.420496 26591 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4d5f17e008314c2587eaa348c6ac8810 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d5f17e008314c2587eaa348c6ac8810" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 46093 } }
I20260805 23:10:11.420677 26591 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4d5f17e008314c2587eaa348c6ac8810 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:10:11.420755 26591 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4d5f17e008314c2587eaa348c6ac8810 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:10:11.420845 26594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:11.405318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39014 (local address 127.25.92.254:46093)
0805 23:10:11.405927 (+ 609us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:11.405949 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:11.405993 (+ 44us) server_negotiation.cc:407] Connection header received
0805 23:10:11.406749 (+ 756us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:11.406769 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:11.407142 (+ 373us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:11.407538 (+ 396us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:11.408469 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:11.409711 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:11.410688 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:11.411268 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:11.411474 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:11.415066 (+ 3592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:11.415072 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:11.415089 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:11.415121 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:11.417244 (+ 2123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:11.418005 (+ 761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:11.418010 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:11.418016 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:11.418175 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:11.418505 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:11.418507 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:11.418509 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:11.418977 (+ 468us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:11.419165 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:11.419451 (+ 286us) server_negotiation.cc:299] Negotiation successful
0805 23:10:11.419807 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":445,"thread_start_us":226,"threads_started":1}
I20260805 23:10:11.420857 26591 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4d5f17e008314c2587eaa348c6ac8810 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:10:11.422027 26591 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4d5f17e008314c2587eaa348c6ac8810 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d5f17e008314c2587eaa348c6ac8810" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 46093 } }
I20260805 23:10:11.422510 26591 leader_election.cc:302] T 00000000000000000000000000000000 P 4d5f17e008314c2587eaa348c6ac8810 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4d5f17e008314c2587eaa348c6ac8810; no voters:
I20260805 23:10:11.422911 26591 leader_election.cc:288] T 00000000000000000000000000000000 P 4d5f17e008314c2587eaa348c6ac8810 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 23:10:11.423182 26596 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4d5f17e008314c2587eaa348c6ac8810 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:10:11.423486 26596 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4d5f17e008314c2587eaa348c6ac8810 [term 1 LEADER]: Becoming Leader. State: Replica: 4d5f17e008314c2587eaa348c6ac8810, State: Running, Role: LEADER
I20260805 23:10:11.423854 26596 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4d5f17e008314c2587eaa348c6ac8810 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d5f17e008314c2587eaa348c6ac8810" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 46093 } }
I20260805 23:10:11.424316 26591 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4d5f17e008314c2587eaa348c6ac8810 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:10:11.425771 26598 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d5f17e008314c2587eaa348c6ac8810 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4d5f17e008314c2587eaa348c6ac8810. Latest consensus state: current_term: 1 leader_uuid: "4d5f17e008314c2587eaa348c6ac8810" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d5f17e008314c2587eaa348c6ac8810" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 46093 } } }
I20260805 23:10:11.425804 26597 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d5f17e008314c2587eaa348c6ac8810 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4d5f17e008314c2587eaa348c6ac8810" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d5f17e008314c2587eaa348c6ac8810" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 46093 } } }
I20260805 23:10:11.426050 26597 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d5f17e008314c2587eaa348c6ac8810 [sys.catalog]: This master's current role is: LEADER
I20260805 23:10:11.426013 26598 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d5f17e008314c2587eaa348c6ac8810 [sys.catalog]: This master's current role is: LEADER
I20260805 23:10:11.426486 26607 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:10:11.429782 26607 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:10:11.435818 26607 catalog_manager.cc:1347] Generated new cluster ID: 8a6231d99a494a928848ce5052ecb32d
I20260805 23:10:11.435915 26607 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:10:11.451097 26607 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:10:11.452142 26607 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:10:11.466133 26607 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4d5f17e008314c2587eaa348c6ac8810: Generated new TSK 0
I20260805 23:10:11.467082 26607 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.92.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.92.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.92.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.193.keytab.
Entry for principal kudu/127.25.92.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.193.keytab.
I20260805 23:10:11.509228 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971406251336-25971-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971406251336-25971-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971406251336-25971-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971406251336-25971-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.92.193:0
--local_ip_for_outbound_sockets=127.25.92.193
--webserver_interface=127.25.92.193
--webserver_port=0
--tserver_master_addrs=127.25.92.254:46093
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.193.keytab
--principal=kudu/127.25.92.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:41251
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:10:11.629905 26618 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:10:11.630244 26618 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:10:11.630322 26618 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:10:11.634588 26618 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:10:11.634696 26618 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:10:11.634791 26618 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.92.193
I20260805 23:10:11.640735 26618 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:41251
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971406251336-25971-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971406251336-25971-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.193.keytab
--principal=kudu/127.25.92.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.92.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971406251336-25971-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.92.193
--webserver_port=0
--tserver_master_addrs=127.25.92.254:46093
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26618
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.92.193
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971406251336-25971-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:10:11.642009 26618 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:10:11.643013 26618 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:10:11.653281 26623 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:11.653314 26626 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:11.653275 26624 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:10:11.653379 26618 server_base.cc:1034] running on GCE node
I20260805 23:10:11.654145 26618 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:10:11.654875 26618 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:10:11.656051 26618 hybrid_clock.cc:648] HybridClock initialized: now 1785971411656030 us; error 36 us; skew 500 ppm
Aug 05 23:10:11 dist-test-slave-ss4n krb5kdc[26510](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971411, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:10:11.659173 26618 init.cc:377] Logged in from keytab as kudu/127.25.92.193@KRBTEST.COM (short username kudu)
I20260805 23:10:11.660097 26618 webserver.cc:460] Webserver started at http://127.25.92.193:40681/ using document root <none> and password file <none>
I20260805 23:10:11.660640 26618 fs_manager.cc:359] Metadata directory not provided
I20260805 23:10:11.660698 26618 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:10:11.660880 26618 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:10:11.663094 26618 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971406251336-25971-0/minicluster-data/ts-0/data/instance:
uuid: "7e861898fa484aeca463b2724e32361a"
format_stamp: "Formatted at 2026-08-05 23:10:11 on dist-test-slave-ss4n"
server_key: "749a7b3fe892e8103577a51945716332"
server_key_iv: "4eabee8d91c24c0cbc72bdf2857bef57"
server_key_version: "encryptionkey@0"
I20260805 23:10:11.663676 26618 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971406251336-25971-0/minicluster-data/ts-0/wal/instance:
uuid: "7e861898fa484aeca463b2724e32361a"
format_stamp: "Formatted at 2026-08-05 23:10:11 on dist-test-slave-ss4n"
server_key: "749a7b3fe892e8103577a51945716332"
server_key_iv: "4eabee8d91c24c0cbc72bdf2857bef57"
server_key_version: "encryptionkey@0"
I20260805 23:10:11.667693 26618 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260805 23:10:11.670490 26633 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:11.671998 26618 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260805 23:10:11.672178 26618 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971406251336-25971-0/minicluster-data/ts-0/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971406251336-25971-0/minicluster-data/ts-0/wal
uuid: "7e861898fa484aeca463b2724e32361a"
format_stamp: "Formatted at 2026-08-05 23:10:11 on dist-test-slave-ss4n"
server_key: "749a7b3fe892e8103577a51945716332"
server_key_iv: "4eabee8d91c24c0cbc72bdf2857bef57"
server_key_version: "encryptionkey@0"
I20260805 23:10:11.672323 26618 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971406251336-25971-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971406251336-25971-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971406251336-25971-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:10:11.690321 26618 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:10:11.694057 26618 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:10:11.694295 26618 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:10:11.694988 26618 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:10:11.695914 26618 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:10:11.695964 26618 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:11.696008 26618 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:10:11.696074 26618 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:11.706158 26618 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.193:36419
I20260805 23:10:11.706184 26747 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.193:36419 every 8 connection(s)
I20260805 23:10:11.707180 26618 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971406251336-25971-0/minicluster-data/ts-0/data/info.pb
I20260805 23:10:11.717183 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 26618
I20260805 23:10:11.717355 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971406251336-25971-0/minicluster-data/ts-0/wal/instance
I20260805 23:10:11.717682 25971 external_mini_cluster.cc:1428] Setting key 5eb05115c2b8c23a1f5d8f336f5b4918
Aug 05 23:10:11 dist-test-slave-ss4n krb5kdc[26510](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971411, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.193@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM
I20260805 23:10:11.723517 26594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:11.709456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.193:37109 (local address 127.25.92.254:46093)
0805 23:10:11.709688 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:11.709692 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:11.710499 (+ 807us) server_negotiation.cc:407] Connection header received
0805 23:10:11.712051 (+ 1552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:11.712054 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:11.712110 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:11.712203 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:11.713649 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:11.714229 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:11.715452 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:11.715753 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:11.715850 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:11.718900 (+ 3050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:11.718903 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:11.718906 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:11.718931 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:11.720573 (+ 1642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:11.721351 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:11.721354 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:11.721355 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:11.721440 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:11.722140 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:11.722143 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:11.722144 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:11.722361 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:11.722446 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:11.723193 (+ 747us) server_negotiation.cc:299] Negotiation successful
0805 23:10:11.723329 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124}
I20260805 23:10:11.724818 26750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:11.709791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:46093 (local address 127.25.92.193:37109)
0805 23:10:11.710328 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:11.710375 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:11.711777 (+ 1402us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:11.712385 (+ 608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:11.712392 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:11.712784 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:11.713379 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:11.713394 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:11.714360 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:11.714363 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:11.715310 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:11.715320 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:11.715948 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:11.715951 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:11.716122 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:11.716826 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:10:11.716846 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:11.718723 (+ 1877us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:10:11.720717 (+ 1994us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:11.720724 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:11.720743 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:11.721203 (+ 460us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:11.721761 (+ 558us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:11.721766 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:11.721770 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:11.721993 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:11.722479 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:10:11.722490 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:10:11.722857 (+ 367us) client_negotiation.cc:769] Sending connection context
0805 23:10:11.723084 (+ 227us) client_negotiation.cc:240] Negotiation successful
0805 23:10:11.723500 (+ 416us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":123,"threads_started":1}
I20260805 23:10:11.726300 26748 heartbeater.cc:347] Connected to a master server at 127.25.92.254:46093
I20260805 23:10:11.726609 26748 heartbeater.cc:464] Registering TS with master...
I20260805 23:10:11.727180 26748 heartbeater.cc:511] Master 127.25.92.254:46093 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.25.92.194@KRBTEST.COM; defaulting to no policy
I20260805 23:10:11.729005 26555 ts_manager.cc:194] Registered new tserver with Master: 7e861898fa484aeca463b2724e32361a (127.25.92.193:36419)
I20260805 23:10:11.730290 26555 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.92.193@KRBTEST.COM'} at 127.25.92.193:37109
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.92.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.92.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.194.keytab.
Entry for principal kudu/127.25.92.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.194.keytab.
I20260805 23:10:11.750124 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971406251336-25971-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971406251336-25971-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971406251336-25971-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971406251336-25971-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.92.194:0
--local_ip_for_outbound_sockets=127.25.92.194
--webserver_interface=127.25.92.194
--webserver_port=0
--tserver_master_addrs=127.25.92.254:46093
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.194.keytab
--principal=kudu/127.25.92.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:41251
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:10:11.866128 26753 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:10:11.866488 26753 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:10:11.866580 26753 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:10:11.870985 26753 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:10:11.871116 26753 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:10:11.871214 26753 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.92.194
I20260805 23:10:11.877514 26753 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:41251
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971406251336-25971-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971406251336-25971-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.194.keytab
--principal=kudu/127.25.92.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.92.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971406251336-25971-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.92.194
--webserver_port=0
--tserver_master_addrs=127.25.92.254:46093
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26753
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.92.194
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971406251336-25971-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:10:11.878839 26753 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:10:11.879892 26753 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:10:11.890522 26758 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:11.890540 26759 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:11.890522 26761 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:10:11.890621 26753 server_base.cc:1034] running on GCE node
I20260805 23:10:11.891225 26753 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:10:11.891974 26753 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:10:11.893188 26753 hybrid_clock.cc:648] HybridClock initialized: now 1785971411893154 us; error 50 us; skew 500 ppm
Aug 05 23:10:11 dist-test-slave-ss4n krb5kdc[26510](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971411, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:10:11.896283 26753 init.cc:377] Logged in from keytab as kudu/127.25.92.194@KRBTEST.COM (short username kudu)
I20260805 23:10:11.897322 26753 webserver.cc:460] Webserver started at http://127.25.92.194:40693/ using document root <none> and password file <none>
I20260805 23:10:11.897943 26753 fs_manager.cc:359] Metadata directory not provided
I20260805 23:10:11.898012 26753 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:10:11.898190 26753 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:10:11.900288 26753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971406251336-25971-0/minicluster-data/ts-1/data/instance:
uuid: "63fe2b8bbb854087971289e9960748c6"
format_stamp: "Formatted at 2026-08-05 23:10:11 on dist-test-slave-ss4n"
server_key: "16d588f51df8e30d88faebe4ddaab406"
server_key_iv: "852432c8897d19d4c70f46739aaecec5"
server_key_version: "encryptionkey@0"
I20260805 23:10:11.900840 26753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971406251336-25971-0/minicluster-data/ts-1/wal/instance:
uuid: "63fe2b8bbb854087971289e9960748c6"
format_stamp: "Formatted at 2026-08-05 23:10:11 on dist-test-slave-ss4n"
server_key: "16d588f51df8e30d88faebe4ddaab406"
server_key_iv: "852432c8897d19d4c70f46739aaecec5"
server_key_version: "encryptionkey@0"
I20260805 23:10:11.904891 26753 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260805 23:10:11.907862 26768 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:11.909322 26753 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260805 23:10:11.909447 26753 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971406251336-25971-0/minicluster-data/ts-1/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971406251336-25971-0/minicluster-data/ts-1/wal
uuid: "63fe2b8bbb854087971289e9960748c6"
format_stamp: "Formatted at 2026-08-05 23:10:11 on dist-test-slave-ss4n"
server_key: "16d588f51df8e30d88faebe4ddaab406"
server_key_iv: "852432c8897d19d4c70f46739aaecec5"
server_key_version: "encryptionkey@0"
I20260805 23:10:11.909709 26753 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971406251336-25971-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971406251336-25971-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971406251336-25971-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:10:11.930573 26753 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:10:11.934048 26753 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:10:11.934265 26753 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:10:11.934906 26753 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:10:11.935781 26753 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:10:11.935861 26753 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:11.935951 26753 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:10:11.935998 26753 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:11.946287 26753 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.194:42397
I20260805 23:10:11.946331 26882 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.194:42397 every 8 connection(s)
I20260805 23:10:11.947256 26753 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971406251336-25971-0/minicluster-data/ts-1/data/info.pb
I20260805 23:10:11.948067 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 26753
I20260805 23:10:11.948154 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971406251336-25971-0/minicluster-data/ts-1/wal/instance
I20260805 23:10:11.948468 25971 external_mini_cluster.cc:1428] Setting key 3cffa2df37d2c927a2d0c1cef7809e2c
Aug 05 23:10:11 dist-test-slave-ss4n krb5kdc[26510](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971411, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.194@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.25.92.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.92.195@KRBTEST.COM" created.
I20260805 23:10:11.961846 26594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:11.949364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.194:33251 (local address 127.25.92.254:46093)
0805 23:10:11.949531 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:11.949535 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:11.950528 (+ 993us) server_negotiation.cc:407] Connection header received
0805 23:10:11.951680 (+ 1152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:11.951684 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:11.951740 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:11.951822 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:11.953001 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:11.953610 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:11.954757 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:11.954986 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:11.955118 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:11.958086 (+ 2968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:11.958089 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:11.958091 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:11.958118 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:11.959641 (+ 1523us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:11.960195 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:11.960198 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:11.960201 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:11.960282 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:11.960650 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:11.960655 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:11.960656 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:11.960855 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:11.960968 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:11.961542 (+ 574us) server_negotiation.cc:299] Negotiation successful
0805 23:10:11.961675 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260805 23:10:11.962760 26885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:11.949864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:46093 (local address 127.25.92.194:33251)
0805 23:10:11.950373 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:11.950426 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:11.951446 (+ 1020us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:11.951993 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:11.952000 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:11.952361 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:11.952848 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:11.952862 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:11.953738 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:11.953747 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:11.954623 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:11.954633 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:11.955165 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:11.955170 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:11.955342 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:11.955979 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:10:11.955996 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:11.957852 (+ 1856us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:10:11.959776 (+ 1924us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:11.959783 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:11.959801 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:11.960083 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:11.960377 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:11.960381 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:11.960384 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:11.960543 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:11.960967 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:10:11.960975 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:10:11.961264 (+ 289us) client_negotiation.cc:769] Sending connection context
0805 23:10:11.961470 (+ 206us) client_negotiation.cc:240] Negotiation successful
0805 23:10:11.961857 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":161,"threads_started":1}
I20260805 23:10:11.964309 26883 heartbeater.cc:347] Connected to a master server at 127.25.92.254:46093
I20260805 23:10:11.964658 26883 heartbeater.cc:464] Registering TS with master...
I20260805 23:10:11.965324 26883 heartbeater.cc:511] Master 127.25.92.254:46093 requested a full tablet report, sending...
I20260805 23:10:11.966797 26555 ts_manager.cc:194] Registered new tserver with Master: 63fe2b8bbb854087971289e9960748c6 (127.25.92.194:42397)
I20260805 23:10:11.967435 26555 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.92.194@KRBTEST.COM'} at 127.25.92.194:33251
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.92.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.195.keytab.
Entry for principal kudu/127.25.92.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.195.keytab.
I20260805 23:10:11.981151 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971406251336-25971-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971406251336-25971-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971406251336-25971-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971406251336-25971-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.92.195:0
--local_ip_for_outbound_sockets=127.25.92.195
--webserver_interface=127.25.92.195
--webserver_port=0
--tserver_master_addrs=127.25.92.254:46093
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.195.keytab
--principal=kudu/127.25.92.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:41251
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:10:12.098038 26888 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:10:12.098335 26888 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:10:12.098381 26888 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:10:12.102816 26888 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:10:12.102890 26888 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:10:12.102993 26888 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.92.195
I20260805 23:10:12.108799 26888 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:41251
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971406251336-25971-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971406251336-25971-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.195.keytab
--principal=kudu/127.25.92.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.92.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971406251336-25971-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.92.195
--webserver_port=0
--tserver_master_addrs=127.25.92.254:46093
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26888
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.92.195
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971406251336-25971-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:10:12.110004 26888 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:10:12.111080 26888 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:10:12.121724 26893 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:12.121728 26896 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:10:12.121951 26888 server_base.cc:1034] running on GCE node
W20260805 23:10:12.121724 26894 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:10:12.122468 26888 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:10:12.123214 26888 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:10:12.124409 26888 hybrid_clock.cc:648] HybridClock initialized: now 1785971412124388 us; error 64 us; skew 500 ppm
Aug 05 23:10:12 dist-test-slave-ss4n krb5kdc[26510](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971412, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:10:12.127893 26888 init.cc:377] Logged in from keytab as kudu/127.25.92.195@KRBTEST.COM (short username kudu)
I20260805 23:10:12.128871 26888 webserver.cc:460] Webserver started at http://127.25.92.195:45453/ using document root <none> and password file <none>
I20260805 23:10:12.129559 26888 fs_manager.cc:359] Metadata directory not provided
I20260805 23:10:12.129657 26888 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:10:12.129884 26888 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:10:12.132102 26888 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971406251336-25971-0/minicluster-data/ts-2/data/instance:
uuid: "cc9a17d2c08745f0820157fa8d970b81"
format_stamp: "Formatted at 2026-08-05 23:10:12 on dist-test-slave-ss4n"
server_key: "772769b14b61c89f9bfd9a1ae131dfa2"
server_key_iv: "65ebbf94493b025190a56f16ab33ebf2"
server_key_version: "encryptionkey@0"
I20260805 23:10:12.132772 26888 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971406251336-25971-0/minicluster-data/ts-2/wal/instance:
uuid: "cc9a17d2c08745f0820157fa8d970b81"
format_stamp: "Formatted at 2026-08-05 23:10:12 on dist-test-slave-ss4n"
server_key: "772769b14b61c89f9bfd9a1ae131dfa2"
server_key_iv: "65ebbf94493b025190a56f16ab33ebf2"
server_key_version: "encryptionkey@0"
I20260805 23:10:12.137058 26888 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260805 23:10:12.140162 26903 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:12.141747 26888 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:10:12.141901 26888 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971406251336-25971-0/minicluster-data/ts-2/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971406251336-25971-0/minicluster-data/ts-2/wal
uuid: "cc9a17d2c08745f0820157fa8d970b81"
format_stamp: "Formatted at 2026-08-05 23:10:12 on dist-test-slave-ss4n"
server_key: "772769b14b61c89f9bfd9a1ae131dfa2"
server_key_iv: "65ebbf94493b025190a56f16ab33ebf2"
server_key_version: "encryptionkey@0"
I20260805 23:10:12.142035 26888 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971406251336-25971-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971406251336-25971-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971406251336-25971-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:10:12.185066 26888 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:10:12.189424 26888 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:10:12.189707 26888 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:10:12.190371 26888 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:10:12.191609 26888 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:10:12.191679 26888 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:12.191746 26888 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:10:12.191781 26888 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:12.203351 26888 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.195:38187
I20260805 23:10:12.203390 27016 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.195:38187 every 8 connection(s)
I20260805 23:10:12.204319 26888 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971406251336-25971-0/minicluster-data/ts-2/data/info.pb
I20260805 23:10:12.209265 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 26888
I20260805 23:10:12.209400 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971406251336-25971-0/minicluster-data/ts-2/wal/instance
I20260805 23:10:12.209794 25971 external_mini_cluster.cc:1428] Setting key 5d0d439b614be2b5b1d7b030cb1bf588
Aug 05 23:10:12 dist-test-slave-ss4n krb5kdc[26510](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971412, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.195@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM
I20260805 23:10:12.219216 26594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:12.206363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.195:59485 (local address 127.25.92.254:46093)
0805 23:10:12.206513 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:12.206517 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:12.207388 (+ 871us) server_negotiation.cc:407] Connection header received
0805 23:10:12.208419 (+ 1031us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:12.208423 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:12.208494 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:12.208596 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:12.209991 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:12.210626 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:12.212016 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:12.212257 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:12.212370 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:12.215254 (+ 2884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:12.215258 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:12.215260 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:12.215298 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:12.216908 (+ 1610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:12.217529 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:12.217533 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:12.217535 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:12.217625 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:12.217998 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:12.218003 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:12.218005 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:12.218224 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:12.218336 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:12.218893 (+ 557us) server_negotiation.cc:299] Negotiation successful
0805 23:10:12.219037 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260805 23:10:12.220072 27019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:12.206750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:46093 (local address 127.25.92.195:59485)
0805 23:10:12.207228 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:12.207271 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:12.208104 (+ 833us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:12.208783 (+ 679us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:12.208790 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:12.209177 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:12.209812 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:12.209827 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:12.210747 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:12.210751 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:12.211895 (+ 1144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:12.211904 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:12.212411 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:12.212415 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:12.212597 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:12.213317 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:10:12.213334 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:12.215055 (+ 1721us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:10:12.217054 (+ 1999us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:12.217061 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:12.217080 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:12.217375 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:12.217728 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:12.217731 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:12.217733 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:12.217893 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:12.218333 (+ 440us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:10:12.218341 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:10:12.218641 (+ 300us) client_negotiation.cc:769] Sending connection context
0805 23:10:12.218854 (+ 213us) client_negotiation.cc:240] Negotiation successful
0805 23:10:12.219206 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":115,"threads_started":1}
I20260805 23:10:12.221320 27017 heartbeater.cc:347] Connected to a master server at 127.25.92.254:46093
I20260805 23:10:12.221679 27017 heartbeater.cc:464] Registering TS with master...
I20260805 23:10:12.222226 27017 heartbeater.cc:511] Master 127.25.92.254:46093 requested a full tablet report, sending...
I20260805 23:10:12.223294 26555 ts_manager.cc:194] Registered new tserver with Master: cc9a17d2c08745f0820157fa8d970b81 (127.25.92.195:38187)
I20260805 23:10:12.223978 26555 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.92.195@KRBTEST.COM'} at 127.25.92.195:59485
I20260805 23:10:12.224125 25971 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260805 23:10:12.236146 26594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:12.228813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35840 (local address 127.25.92.254:46093)
0805 23:10:12.229013 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:12.229022 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:12.229088 (+ 66us) server_negotiation.cc:407] Connection header received
0805 23:10:12.229203 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:12.229205 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:12.229243 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:12.229311 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:12.230123 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:12.230647 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:12.231436 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:12.231656 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:12.231736 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:12.233192 (+ 1456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:12.233195 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:12.233197 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:12.233219 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:12.234562 (+ 1343us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:12.234941 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:12.234944 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:12.234945 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:12.235016 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:12.235241 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:12.235243 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:12.235244 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:12.235451 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:12.235559 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:12.235824 (+ 265us) server_negotiation.cc:299] Negotiation successful
0805 23:10:12.235918 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260805 23:10:12.242532 26555 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35840:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260805 23:10:12.246379 26555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260805 23:10:12.263473 27028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:12.256845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.193:36419 (local address 127.0.0.1:34796)
0805 23:10:12.257725 (+ 880us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:12.257782 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:12.257931 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:12.258780 (+ 849us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:12.258787 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:12.258829 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:10:12.259116 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:12.259127 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:12.260145 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:12.260150 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:12.261847 (+ 1697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:12.261858 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:12.262928 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:12.262933 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:12.263042 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:12.263091 (+ 49us) client_negotiation.cc:769] Sending connection context
0805 23:10:12.263185 (+ 94us) client_negotiation.cc:240] Negotiation successful
0805 23:10:12.263308 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":696,"mutex_wait_us":85,"spinlock_wait_cycles":7680,"thread_start_us":100,"threads_started":1}
I20260805 23:10:12.263520 27030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:12.256973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34796 (local address 127.25.92.193:36419)
0805 23:10:12.257669 (+ 696us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:12.257679 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:12.258118 (+ 439us) server_negotiation.cc:407] Connection header received
0805 23:10:12.258213 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:12.258220 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:12.258438 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:12.258592 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:10:12.259262 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:12.260022 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:12.261978 (+ 1956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:12.262793 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:12.262918 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:12.263158 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:12.263241 (+ 83us) server_negotiation.cc:299] Negotiation successful
0805 23:10:12.263361 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":573,"thread_start_us":93,"threads_started":1}
I20260805 23:10:12.264401 27032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:12.256898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.195:38187 (local address 127.0.0.1:34580)
0805 23:10:12.257965 (+ 1067us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:12.257980 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:12.258071 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:12.258780 (+ 709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:12.258787 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:12.258828 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:10:12.259190 (+ 362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:12.259200 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:12.261991 (+ 2791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:12.261994 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:12.262840 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:12.262849 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:12.263952 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:12.263956 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:12.264068 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:12.264095 (+ 27us) client_negotiation.cc:769] Sending connection context
0805 23:10:12.264177 (+ 82us) client_negotiation.cc:240] Negotiation successful
0805 23:10:12.264251 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":994,"mutex_wait_us":164,"spinlock_wait_cycles":6784,"thread_start_us":113,"threads_started":1}
I20260805 23:10:12.265156 27029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:12.257421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.194:42397 (local address 127.0.0.1:53476)
0805 23:10:12.258447 (+ 1026us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:12.258461 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:12.258561 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:12.258984 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:12.258987 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:12.259004 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:10:12.259189 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:12.259217 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:12.260868 (+ 1651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:12.260871 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:12.262202 (+ 1331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:12.262211 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:12.264681 (+ 2470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:12.264687 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:12.264827 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:12.264859 (+ 32us) client_negotiation.cc:769] Sending connection context
0805 23:10:12.264946 (+ 87us) client_negotiation.cc:240] Negotiation successful
0805 23:10:12.265034 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":963,"mutex_wait_us":25,"thread_start_us":143,"threads_started":1}
I20260805 23:10:12.265247 27033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:12.257907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53476 (local address 127.25.92.194:42397)
0805 23:10:12.258295 (+ 388us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:12.258308 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:12.258657 (+ 349us) server_negotiation.cc:407] Connection header received
0805 23:10:12.258722 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:12.258726 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:12.258852 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:12.259001 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:10:12.259337 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:12.260669 (+ 1332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:12.262332 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:12.263274 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:12.263381 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:12.263512 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:12.264991 (+ 1479us) server_negotiation.cc:299] Negotiation successful
0805 23:10:12.265111 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":111,"threads_started":1}
I20260805 23:10:12.267107 26682 tablet_service.cc:1467] Processing CreateTablet for tablet b87e497992264d969dc616a14732a77b (DEFAULT_TABLE table=test-table [id=da63b4e756a94d4c8bbe62358d57471a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:10:12.267628 26816 tablet_service.cc:1467] Processing CreateTablet for tablet b87e497992264d969dc616a14732a77b (DEFAULT_TABLE table=test-table [id=da63b4e756a94d4c8bbe62358d57471a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:10:12.268265 26682 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b87e497992264d969dc616a14732a77b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:10:12.268680 26816 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b87e497992264d969dc616a14732a77b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:10:12.273463 27031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:12.257289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34580 (local address 127.25.92.195:38187)
0805 23:10:12.257676 (+ 387us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:12.257687 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:12.258177 (+ 490us) server_negotiation.cc:407] Connection header received
0805 23:10:12.258253 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:12.258258 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:12.258437 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:12.258593 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:10:12.259336 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:12.260669 (+ 1333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:12.263008 (+ 2339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:12.263811 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:12.263916 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:12.264008 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:12.264330 (+ 322us) server_negotiation.cc:299] Negotiation successful
0805 23:10:12.264432 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":91,"threads_started":1}
I20260805 23:10:12.275125 27034 tablet_bootstrap.cc:492] T b87e497992264d969dc616a14732a77b P 7e861898fa484aeca463b2724e32361a: Bootstrap starting.
I20260805 23:10:12.276485 27035 tablet_bootstrap.cc:492] T b87e497992264d969dc616a14732a77b P 63fe2b8bbb854087971289e9960748c6: Bootstrap starting.
I20260805 23:10:12.277107 26951 tablet_service.cc:1467] Processing CreateTablet for tablet b87e497992264d969dc616a14732a77b (DEFAULT_TABLE table=test-table [id=da63b4e756a94d4c8bbe62358d57471a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:10:12.277634 27034 tablet_bootstrap.cc:654] T b87e497992264d969dc616a14732a77b P 7e861898fa484aeca463b2724e32361a: Neither blocks nor log segments found. Creating new log.
I20260805 23:10:12.278273 26951 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b87e497992264d969dc616a14732a77b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:10:12.278502 27034 log.cc:824] T b87e497992264d969dc616a14732a77b P 7e861898fa484aeca463b2724e32361a: Log is configured to *not* fsync() on all Append() calls
I20260805 23:10:12.279441 27035 tablet_bootstrap.cc:654] T b87e497992264d969dc616a14732a77b P 63fe2b8bbb854087971289e9960748c6: Neither blocks nor log segments found. Creating new log.
I20260805 23:10:12.280341 27035 log.cc:824] T b87e497992264d969dc616a14732a77b P 63fe2b8bbb854087971289e9960748c6: Log is configured to *not* fsync() on all Append() calls
I20260805 23:10:12.280983 27034 tablet_bootstrap.cc:492] T b87e497992264d969dc616a14732a77b P 7e861898fa484aeca463b2724e32361a: No bootstrap required, opened a new log
I20260805 23:10:12.281208 27034 ts_tablet_manager.cc:1397] T b87e497992264d969dc616a14732a77b P 7e861898fa484aeca463b2724e32361a: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260805 23:10:12.282220 27035 tablet_bootstrap.cc:492] T b87e497992264d969dc616a14732a77b P 63fe2b8bbb854087971289e9960748c6: No bootstrap required, opened a new log
I20260805 23:10:12.282426 27035 ts_tablet_manager.cc:1397] T b87e497992264d969dc616a14732a77b P 63fe2b8bbb854087971289e9960748c6: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260805 23:10:12.285564 27035 raft_consensus.cc:348] T b87e497992264d969dc616a14732a77b P 63fe2b8bbb854087971289e9960748c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e861898fa484aeca463b2724e32361a" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 36419 } } peers { permanent_uuid: "cc9a17d2c08745f0820157fa8d970b81" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38187 } } peers { permanent_uuid: "63fe2b8bbb854087971289e9960748c6" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 42397 } }
I20260805 23:10:12.285843 27035 raft_consensus.cc:374] T b87e497992264d969dc616a14732a77b P 63fe2b8bbb854087971289e9960748c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:10:12.285897 27035 raft_consensus.cc:729] T b87e497992264d969dc616a14732a77b P 63fe2b8bbb854087971289e9960748c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63fe2b8bbb854087971289e9960748c6, State: Initialized, Role: FOLLOWER
I20260805 23:10:12.286381 27035 consensus_queue.cc:260] T b87e497992264d969dc616a14732a77b P 63fe2b8bbb854087971289e9960748c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e861898fa484aeca463b2724e32361a" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 36419 } } peers { permanent_uuid: "cc9a17d2c08745f0820157fa8d970b81" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38187 } } peers { permanent_uuid: "63fe2b8bbb854087971289e9960748c6" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 42397 } }
I20260805 23:10:12.286573 27034 raft_consensus.cc:348] T b87e497992264d969dc616a14732a77b P 7e861898fa484aeca463b2724e32361a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e861898fa484aeca463b2724e32361a" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 36419 } } peers { permanent_uuid: "cc9a17d2c08745f0820157fa8d970b81" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38187 } } peers { permanent_uuid: "63fe2b8bbb854087971289e9960748c6" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 42397 } }
I20260805 23:10:12.286839 27034 raft_consensus.cc:374] T b87e497992264d969dc616a14732a77b P 7e861898fa484aeca463b2724e32361a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:10:12.286896 27034 raft_consensus.cc:729] T b87e497992264d969dc616a14732a77b P 7e861898fa484aeca463b2724e32361a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e861898fa484aeca463b2724e32361a, State: Initialized, Role: FOLLOWER
I20260805 23:10:12.287423 27035 ts_tablet_manager.cc:1428] T b87e497992264d969dc616a14732a77b P 63fe2b8bbb854087971289e9960748c6: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260805 23:10:12.287427 27034 consensus_queue.cc:260] T b87e497992264d969dc616a14732a77b P 7e861898fa484aeca463b2724e32361a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e861898fa484aeca463b2724e32361a" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 36419 } } peers { permanent_uuid: "cc9a17d2c08745f0820157fa8d970b81" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38187 } } peers { permanent_uuid: "63fe2b8bbb854087971289e9960748c6" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 42397 } }
I20260805 23:10:12.287856 26883 heartbeater.cc:503] Master 127.25.92.254:46093 was elected leader, sending a full tablet report...
I20260805 23:10:12.288167 27038 tablet_bootstrap.cc:492] T b87e497992264d969dc616a14732a77b P cc9a17d2c08745f0820157fa8d970b81: Bootstrap starting.
I20260805 23:10:12.288247 26748 heartbeater.cc:503] Master 127.25.92.254:46093 was elected leader, sending a full tablet report...
I20260805 23:10:12.288442 27034 ts_tablet_manager.cc:1428] T b87e497992264d969dc616a14732a77b P 7e861898fa484aeca463b2724e32361a: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260805 23:10:12.290693 27038 tablet_bootstrap.cc:654] T b87e497992264d969dc616a14732a77b P cc9a17d2c08745f0820157fa8d970b81: Neither blocks nor log segments found. Creating new log.
I20260805 23:10:12.292172 27038 log.cc:824] T b87e497992264d969dc616a14732a77b P cc9a17d2c08745f0820157fa8d970b81: Log is configured to *not* fsync() on all Append() calls
I20260805 23:10:12.294503 27038 tablet_bootstrap.cc:492] T b87e497992264d969dc616a14732a77b P cc9a17d2c08745f0820157fa8d970b81: No bootstrap required, opened a new log
I20260805 23:10:12.294754 27038 ts_tablet_manager.cc:1397] T b87e497992264d969dc616a14732a77b P cc9a17d2c08745f0820157fa8d970b81: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260805 23:10:12.298532 27038 raft_consensus.cc:348] T b87e497992264d969dc616a14732a77b P cc9a17d2c08745f0820157fa8d970b81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e861898fa484aeca463b2724e32361a" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 36419 } } peers { permanent_uuid: "cc9a17d2c08745f0820157fa8d970b81" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38187 } } peers { permanent_uuid: "63fe2b8bbb854087971289e9960748c6" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 42397 } }
I20260805 23:10:12.298836 27038 raft_consensus.cc:374] T b87e497992264d969dc616a14732a77b P cc9a17d2c08745f0820157fa8d970b81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:10:12.298913 27038 raft_consensus.cc:729] T b87e497992264d969dc616a14732a77b P cc9a17d2c08745f0820157fa8d970b81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc9a17d2c08745f0820157fa8d970b81, State: Initialized, Role: FOLLOWER
I20260805 23:10:12.299417 27038 consensus_queue.cc:260] T b87e497992264d969dc616a14732a77b P cc9a17d2c08745f0820157fa8d970b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e861898fa484aeca463b2724e32361a" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 36419 } } peers { permanent_uuid: "cc9a17d2c08745f0820157fa8d970b81" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38187 } } peers { permanent_uuid: "63fe2b8bbb854087971289e9960748c6" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 42397 } }
I20260805 23:10:12.300161 27017 heartbeater.cc:503] Master 127.25.92.254:46093 was elected leader, sending a full tablet report...
I20260805 23:10:12.300396 27038 ts_tablet_manager.cc:1428] T b87e497992264d969dc616a14732a77b P cc9a17d2c08745f0820157fa8d970b81: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
W20260805 23:10:12.448431 26884 tablet.cc:2369] T b87e497992264d969dc616a14732a77b P 63fe2b8bbb854087971289e9960748c6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260805 23:10:12.455454 27018 tablet.cc:2369] T b87e497992264d969dc616a14732a77b P cc9a17d2c08745f0820157fa8d970b81: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260805 23:10:12.457422 27042 raft_consensus.cc:482] T b87e497992264d969dc616a14732a77b P cc9a17d2c08745f0820157fa8d970b81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 23:10:12.457628 27042 raft_consensus.cc:504] T b87e497992264d969dc616a14732a77b P cc9a17d2c08745f0820157fa8d970b81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e861898fa484aeca463b2724e32361a" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 36419 } } peers { permanent_uuid: "cc9a17d2c08745f0820157fa8d970b81" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38187 } } peers { permanent_uuid: "63fe2b8bbb854087971289e9960748c6" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 42397 } }
W20260805 23:10:12.458765 26749 tablet.cc:2369] T b87e497992264d969dc616a14732a77b P 7e861898fa484aeca463b2724e32361a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260805 23:10:12.459329 27042 leader_election.cc:288] T b87e497992264d969dc616a14732a77b P cc9a17d2c08745f0820157fa8d970b81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e861898fa484aeca463b2724e32361a (127.25.92.193:36419), 63fe2b8bbb854087971289e9960748c6 (127.25.92.194:42397)
I20260805 23:10:12.464116 27033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:12.459369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.195:53553 (local address 127.25.92.194:42397)
0805 23:10:12.459501 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:12.459507 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:12.459522 (+ 15us) server_negotiation.cc:407] Connection header received
0805 23:10:12.459578 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:12.459581 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:12.459701 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:12.459799 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:10:12.460149 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:12.460914 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:12.462706 (+ 1792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:12.463352 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:12.463481 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:12.463531 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:12.463879 (+ 348us) server_negotiation.cc:299] Negotiation successful
0805 23:10:12.463955 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260805 23:10:12.464102 27030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:12.459428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.195:53531 (local address 127.25.92.193:36419)
0805 23:10:12.459582 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:12.459592 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:12.459975 (+ 383us) server_negotiation.cc:407] Connection header received
0805 23:10:12.460068 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:12.460071 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:12.460147 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:12.460225 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:10:12.460499 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:12.461183 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:12.462706 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:12.463362 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:12.463481 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:12.463531 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:12.463879 (+ 348us) server_negotiation.cc:299] Negotiation successful
0805 23:10:12.463955 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260805 23:10:12.464134 27043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:12.459192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.193:36419 (local address 127.25.92.195:53531)
0805 23:10:12.459873 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:12.459886 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:12.459989 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:12.460241 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:12.460243 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:12.460258 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:10:12.460382 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:12.460388 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:12.461277 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:12.461280 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:12.462579 (+ 1299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:12.462589 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:12.463503 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:12.463506 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:12.463710 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:12.463752 (+ 42us) client_negotiation.cc:769] Sending connection context
0805 23:10:12.463832 (+ 80us) client_negotiation.cc:240] Negotiation successful
0805 23:10:12.463952 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":622,"mutex_wait_us":431,"thread_start_us":129,"threads_started":1}
I20260805 23:10:12.464134 27019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:12.459178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.194:42397 (local address 127.25.92.195:53553)
0805 23:10:12.459316 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:12.459332 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:12.459436 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:12.459832 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:12.459836 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:12.459858 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:10:12.460017 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:12.460027 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:12.461036 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:12.461038 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:12.462521 (+ 1483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:12.462533 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:12.463488 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:12.463492 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:12.463713 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:12.463749 (+ 36us) client_negotiation.cc:769] Sending connection context
0805 23:10:12.463832 (+ 83us) client_negotiation.cc:240] Negotiation successful
0805 23:10:12.463958 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47,"mutex_wait_us":183}
I20260805 23:10:12.465216 26702 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b87e497992264d969dc616a14732a77b" candidate_uuid: "cc9a17d2c08745f0820157fa8d970b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e861898fa484aeca463b2724e32361a" is_pre_election: true
I20260805 23:10:12.465619 26702 raft_consensus.cc:2454] T b87e497992264d969dc616a14732a77b P 7e861898fa484aeca463b2724e32361a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc9a17d2c08745f0820157fa8d970b81 in term 0.
I20260805 23:10:12.466176 26904 leader_election.cc:302] T b87e497992264d969dc616a14732a77b P cc9a17d2c08745f0820157fa8d970b81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e861898fa484aeca463b2724e32361a, cc9a17d2c08745f0820157fa8d970b81; no voters:
I20260805 23:10:12.466426 27042 raft_consensus.cc:2781] T b87e497992264d969dc616a14732a77b P cc9a17d2c08745f0820157fa8d970b81 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260805 23:10:12.466567 27042 raft_consensus.cc:482] T b87e497992264d969dc616a14732a77b P cc9a17d2c08745f0820157fa8d970b81 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260805 23:10:12.466634 27042 raft_consensus.cc:3037] T b87e497992264d969dc616a14732a77b P cc9a17d2c08745f0820157fa8d970b81 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:10:12.466720 26836 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b87e497992264d969dc616a14732a77b" candidate_uuid: "cc9a17d2c08745f0820157fa8d970b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63fe2b8bbb854087971289e9960748c6" is_pre_election: true
I20260805 23:10:12.467020 26836 raft_consensus.cc:2454] T b87e497992264d969dc616a14732a77b P 63fe2b8bbb854087971289e9960748c6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc9a17d2c08745f0820157fa8d970b81 in term 0.
I20260805 23:10:12.467828 27042 raft_consensus.cc:504] T b87e497992264d969dc616a14732a77b P cc9a17d2c08745f0820157fa8d970b81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e861898fa484aeca463b2724e32361a" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 36419 } } peers { permanent_uuid: "cc9a17d2c08745f0820157fa8d970b81" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38187 } } peers { permanent_uuid: "63fe2b8bbb854087971289e9960748c6" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 42397 } }
I20260805 23:10:12.468200 27042 leader_election.cc:288] T b87e497992264d969dc616a14732a77b P cc9a17d2c08745f0820157fa8d970b81 [CANDIDATE]: Term 1 election: Requested vote from peers 7e861898fa484aeca463b2724e32361a (127.25.92.193:36419), 63fe2b8bbb854087971289e9960748c6 (127.25.92.194:42397)
I20260805 23:10:12.468607 26702 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b87e497992264d969dc616a14732a77b" candidate_uuid: "cc9a17d2c08745f0820157fa8d970b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e861898fa484aeca463b2724e32361a"
I20260805 23:10:12.468607 26836 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b87e497992264d969dc616a14732a77b" candidate_uuid: "cc9a17d2c08745f0820157fa8d970b81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63fe2b8bbb854087971289e9960748c6"
I20260805 23:10:12.468734 26836 raft_consensus.cc:3037] T b87e497992264d969dc616a14732a77b P 63fe2b8bbb854087971289e9960748c6 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:10:12.468734 26702 raft_consensus.cc:3037] T b87e497992264d969dc616a14732a77b P 7e861898fa484aeca463b2724e32361a [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:10:12.470275 26836 raft_consensus.cc:2454] T b87e497992264d969dc616a14732a77b P 63fe2b8bbb854087971289e9960748c6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc9a17d2c08745f0820157fa8d970b81 in term 1.
I20260805 23:10:12.470275 26702 raft_consensus.cc:2454] T b87e497992264d969dc616a14732a77b P 7e861898fa484aeca463b2724e32361a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc9a17d2c08745f0820157fa8d970b81 in term 1.
I20260805 23:10:12.470659 26904 leader_election.cc:302] T b87e497992264d969dc616a14732a77b P cc9a17d2c08745f0820157fa8d970b81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 63fe2b8bbb854087971289e9960748c6, 7e861898fa484aeca463b2724e32361a, cc9a17d2c08745f0820157fa8d970b81; no voters:
I20260805 23:10:12.470932 27042 raft_consensus.cc:2781] T b87e497992264d969dc616a14732a77b P cc9a17d2c08745f0820157fa8d970b81 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:10:12.471128 27042 raft_consensus.cc:686] T b87e497992264d969dc616a14732a77b P cc9a17d2c08745f0820157fa8d970b81 [term 1 LEADER]: Becoming Leader. State: Replica: cc9a17d2c08745f0820157fa8d970b81, State: Running, Role: LEADER
I20260805 23:10:12.471518 27042 consensus_queue.cc:237] T b87e497992264d969dc616a14732a77b P cc9a17d2c08745f0820157fa8d970b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e861898fa484aeca463b2724e32361a" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 36419 } } peers { permanent_uuid: "cc9a17d2c08745f0820157fa8d970b81" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38187 } } peers { permanent_uuid: "63fe2b8bbb854087971289e9960748c6" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 42397 } }
I20260805 23:10:12.475445 26555 catalog_manager.cc:5495] T b87e497992264d969dc616a14732a77b P cc9a17d2c08745f0820157fa8d970b81 reported cstate change: term changed from 0 to 1, leader changed from <none> to cc9a17d2c08745f0820157fa8d970b81 (127.25.92.195). New cstate: current_term: 1 leader_uuid: "cc9a17d2c08745f0820157fa8d970b81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e861898fa484aeca463b2724e32361a" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 36419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cc9a17d2c08745f0820157fa8d970b81" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38187 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "63fe2b8bbb854087971289e9960748c6" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 42397 } health_report { overall_health: UNKNOWN } } }
Aug 05 23:10:12 dist-test-slave-ss4n krb5kdc[26510](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971411, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.92.193@KRBTEST.COM
I20260805 23:10:12.505409 27030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:12.494369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34810 (local address 127.25.92.193:36419)
0805 23:10:12.494573 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:12.494579 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:12.494745 (+ 166us) server_negotiation.cc:407] Connection header received
0805 23:10:12.494929 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:12.494940 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:12.494987 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:12.495105 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:12.495911 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:12.496489 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:12.497540 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:12.497778 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:12.497903 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:12.500797 (+ 2894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:12.500815 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:12.500833 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:12.500865 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:12.503151 (+ 2286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:12.503756 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:12.503762 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:12.503764 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:12.503880 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:12.504239 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:12.504244 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:12.504246 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:12.504562 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:12.504798 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:12.505055 (+ 257us) server_negotiation.cc:299] Negotiation successful
0805 23:10:12.505209 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
Aug 05 23:10:12 dist-test-slave-ss4n krb5kdc[26510](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971412, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 05 23:10:12 dist-test-slave-ss4n krb5kdc[26510](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971412, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.25.92.193@KRBTEST.COM
I20260805 23:10:12.535513 27030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:12.525891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34818 (local address 127.25.92.193:36419)
0805 23:10:12.526073 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:12.526078 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:12.526246 (+ 168us) server_negotiation.cc:407] Connection header received
0805 23:10:12.526352 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:12.526355 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:12.526403 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:12.526476 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:12.527256 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:12.527799 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:12.528710 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:12.528939 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:12.529021 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:12.531777 (+ 2756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:12.531780 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:12.531783 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:12.531813 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:12.533401 (+ 1588us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:12.534013 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:12.534016 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:12.534017 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:12.534110 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:12.534450 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:12.534452 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:12.534453 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:12.534687 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:12.534794 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:12.535133 (+ 339us) server_negotiation.cc:299] Negotiation successful
0805 23:10:12.535308 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
Aug 05 23:10:12 dist-test-slave-ss4n krb5kdc[26510](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971412, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 05 23:10:12 dist-test-slave-ss4n krb5kdc[26510](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971412, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.92.193@KRBTEST.COM
I20260805 23:10:12.565083 27030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:12.555695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34834 (local address 127.25.92.193:36419)
0805 23:10:12.555875 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:12.555880 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:12.556088 (+ 208us) server_negotiation.cc:407] Connection header received
0805 23:10:12.556281 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:12.556285 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:12.556345 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:12.556448 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:12.557189 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:12.557783 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:12.558662 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:12.558888 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:12.559003 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:12.561704 (+ 2701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:12.561707 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:12.561709 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:12.561741 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:12.563272 (+ 1531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:12.563774 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:12.563778 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:12.563780 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:12.563871 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:12.564179 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:12.564183 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:12.564185 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:12.564379 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:12.564521 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:12.564778 (+ 257us) server_negotiation.cc:299] Negotiation successful
0805 23:10:12.564908 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
W20260805 23:10:12.567101 26662 tablet_service.cc:2908] Rejecting scan request for tablet b87e497992264d969dc616a14732a77b: Uninitialized: clean time has not yet been initialized
I20260805 23:10:12.570339 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 26618
I20260805 23:10:12.577899 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 26753
I20260805 23:10:12.585041 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 26888
I20260805 23:10:12.592006 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 26523
2026-08-05T23:10:12Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2730 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 23:10:12 dist-test-slave-ss4n krb5kdc[27058](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 23:10:12 dist-test-slave-ss4n krb5kdc[27058](info): set up 2 sockets
Aug 05 23:10:12 dist-test-slave-ss4n krb5kdc[27058](info): commencing operation
krb5kdc: starting...
W20260805 23:10:13.661782 25971 mini_kdc.cc:121] Time spent starting KDC: real 1.045s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/krb5kdc/test-user.keytab.
Aug 05 23:10:13 dist-test-slave-ss4n krb5kdc[27058](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971413, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T23:10:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:10:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.92.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.92.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.92.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab.
Entry for principal kudu/127.25.92.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab.
I20260805 23:10:13.796746 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.254:38411
--webserver_interface=127.25.92.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab
--principal=kudu/127.25.92.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:46793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.92.254:38411
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:10:13.909739 27071 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:10:13.910039 27071 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:10:13.910082 27071 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:10:13.914994 27071 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:10:13.915104 27071 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260805 23:10:13.915131 27071 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:10:13.915159 27071 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:10:13.915203 27071 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:10:13.915232 27071 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:10:13.923075 27071 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:46793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.92.254:38411
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab
--principal=kudu/127.25.92.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.254:38411
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.92.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27071
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:10:13.924307 27071 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:10:13.925387 27071 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:10:13.934725 27078 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:13.934731 27080 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:13.934725 27077 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:10:13.934772 27071 server_base.cc:1034] running on GCE node
I20260805 23:10:13.935534 27071 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:10:13.936774 27071 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:10:13.937968 27071 hybrid_clock.cc:648] HybridClock initialized: now 1785971413937933 us; error 54 us; skew 500 ppm
Aug 05 23:10:13 dist-test-slave-ss4n krb5kdc[27058](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971413, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:10:13.940879 27071 init.cc:377] Logged in from keytab as kudu/127.25.92.254@KRBTEST.COM (short username kudu)
I20260805 23:10:13.941920 27071 webserver.cc:460] Webserver started at http://127.25.92.254:43841/ using document root <none> and password file <none>
I20260805 23:10:13.942529 27071 fs_manager.cc:359] Metadata directory not provided
I20260805 23:10:13.942633 27071 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:10:13.942898 27071 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:10:13.945153 27071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/minicluster-data/master-0/data/instance:
uuid: "94ae863b48d343ceb1a8d8b53a9666f7"
format_stamp: "Formatted at 2026-08-05 23:10:13 on dist-test-slave-ss4n"
server_key: "96b1201bb13c1f89278ff46c51f5da24"
server_key_iv: "eb235d34ed3901febb82ad9d25fedd93"
server_key_version: "encryptionkey@0"
I20260805 23:10:13.945796 27071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/minicluster-data/master-0/wal/instance:
uuid: "94ae863b48d343ceb1a8d8b53a9666f7"
format_stamp: "Formatted at 2026-08-05 23:10:13 on dist-test-slave-ss4n"
server_key: "96b1201bb13c1f89278ff46c51f5da24"
server_key_iv: "eb235d34ed3901febb82ad9d25fedd93"
server_key_version: "encryptionkey@0"
I20260805 23:10:13.950049 27071 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260805 23:10:13.952574 27087 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:13.954016 27071 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:10:13.954173 27071 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/minicluster-data/master-0/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/minicluster-data/master-0/wal
uuid: "94ae863b48d343ceb1a8d8b53a9666f7"
format_stamp: "Formatted at 2026-08-05 23:10:13 on dist-test-slave-ss4n"
server_key: "96b1201bb13c1f89278ff46c51f5da24"
server_key_iv: "eb235d34ed3901febb82ad9d25fedd93"
server_key_version: "encryptionkey@0"
I20260805 23:10:13.954314 27071 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:10:13.969112 27071 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:10:13.972797 27071 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:10:13.973045 27071 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:10:13.982223 27071 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.254:38411
I20260805 23:10:13.982254 27150 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.254:38411 every 8 connection(s)
I20260805 23:10:13.983227 27071 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
I20260805 23:10:13.983644 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 27071
I20260805 23:10:13.983791 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/minicluster-data/master-0/wal/instance
I20260805 23:10:13.984086 25971 external_mini_cluster.cc:1428] Setting key bc9b0a319b1635a30da5de467bdff00e
I20260805 23:10:13.986505 27151 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:10:13.992444 27151 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 94ae863b48d343ceb1a8d8b53a9666f7: Bootstrap starting.
Aug 05 23:10:13 dist-test-slave-ss4n krb5kdc[27058](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971413, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM
I20260805 23:10:13.994984 27151 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 94ae863b48d343ceb1a8d8b53a9666f7: Neither blocks nor log segments found. Creating new log.
I20260805 23:10:13.996289 27151 log.cc:824] T 00000000000000000000000000000000 P 94ae863b48d343ceb1a8d8b53a9666f7: Log is configured to *not* fsync() on all Append() calls
I20260805 23:10:13.998402 27151 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 94ae863b48d343ceb1a8d8b53a9666f7: No bootstrap required, opened a new log
I20260805 23:10:13.999636 27154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:13.985663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36254 (local address 127.25.92.254:38411)
0805 23:10:13.986119 (+ 456us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:13.986139 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:13.986178 (+ 39us) server_negotiation.cc:407] Connection header received
0805 23:10:13.986980 (+ 802us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:13.986999 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:13.987352 (+ 353us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:13.987727 (+ 375us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:13.988647 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:13.989800 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:13.990748 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:13.991265 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:13.991457 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:13.994354 (+ 2897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:13.994359 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:13.994374 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:13.994406 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:13.996537 (+ 2131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:13.997038 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:13.997050 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:13.997055 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:13.997185 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:13.997524 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:13.997528 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:13.997530 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:13.997987 (+ 457us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:13.998219 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:13.998546 (+ 327us) server_negotiation.cc:299] Negotiation successful
0805 23:10:13.998866 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":131,"threads_started":1}
I20260805 23:10:14.002249 27151 raft_consensus.cc:348] T 00000000000000000000000000000000 P 94ae863b48d343ceb1a8d8b53a9666f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94ae863b48d343ceb1a8d8b53a9666f7" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 38411 } }
I20260805 23:10:14.002521 27151 raft_consensus.cc:374] T 00000000000000000000000000000000 P 94ae863b48d343ceb1a8d8b53a9666f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:10:14.002609 27151 raft_consensus.cc:729] T 00000000000000000000000000000000 P 94ae863b48d343ceb1a8d8b53a9666f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94ae863b48d343ceb1a8d8b53a9666f7, State: Initialized, Role: FOLLOWER
I20260805 23:10:14.003085 27151 consensus_queue.cc:260] T 00000000000000000000000000000000 P 94ae863b48d343ceb1a8d8b53a9666f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94ae863b48d343ceb1a8d8b53a9666f7" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 38411 } }
I20260805 23:10:14.003257 27151 raft_consensus.cc:388] T 00000000000000000000000000000000 P 94ae863b48d343ceb1a8d8b53a9666f7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:10:14.003335 27151 raft_consensus.cc:482] T 00000000000000000000000000000000 P 94ae863b48d343ceb1a8d8b53a9666f7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:10:14.003438 27151 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 94ae863b48d343ceb1a8d8b53a9666f7 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:10:14.004441 27151 raft_consensus.cc:504] T 00000000000000000000000000000000 P 94ae863b48d343ceb1a8d8b53a9666f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94ae863b48d343ceb1a8d8b53a9666f7" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 38411 } }
I20260805 23:10:14.004833 27151 leader_election.cc:302] T 00000000000000000000000000000000 P 94ae863b48d343ceb1a8d8b53a9666f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 94ae863b48d343ceb1a8d8b53a9666f7; no voters:
I20260805 23:10:14.005255 27151 leader_election.cc:288] T 00000000000000000000000000000000 P 94ae863b48d343ceb1a8d8b53a9666f7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 23:10:14.005332 27156 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 94ae863b48d343ceb1a8d8b53a9666f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:10:14.005632 27156 raft_consensus.cc:686] T 00000000000000000000000000000000 P 94ae863b48d343ceb1a8d8b53a9666f7 [term 1 LEADER]: Becoming Leader. State: Replica: 94ae863b48d343ceb1a8d8b53a9666f7, State: Running, Role: LEADER
I20260805 23:10:14.005941 27156 consensus_queue.cc:237] T 00000000000000000000000000000000 P 94ae863b48d343ceb1a8d8b53a9666f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94ae863b48d343ceb1a8d8b53a9666f7" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 38411 } }
I20260805 23:10:14.006722 27151 sys_catalog.cc:564] T 00000000000000000000000000000000 P 94ae863b48d343ceb1a8d8b53a9666f7 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:10:14.007912 27157 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94ae863b48d343ceb1a8d8b53a9666f7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "94ae863b48d343ceb1a8d8b53a9666f7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94ae863b48d343ceb1a8d8b53a9666f7" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 38411 } } }
I20260805 23:10:14.008076 27157 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94ae863b48d343ceb1a8d8b53a9666f7 [sys.catalog]: This master's current role is: LEADER
I20260805 23:10:14.008086 27158 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94ae863b48d343ceb1a8d8b53a9666f7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 94ae863b48d343ceb1a8d8b53a9666f7. Latest consensus state: current_term: 1 leader_uuid: "94ae863b48d343ceb1a8d8b53a9666f7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94ae863b48d343ceb1a8d8b53a9666f7" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 38411 } } }
I20260805 23:10:14.008363 27158 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94ae863b48d343ceb1a8d8b53a9666f7 [sys.catalog]: This master's current role is: LEADER
W20260805 23:10:14.011646 27171 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 94ae863b48d343ceb1a8d8b53a9666f7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260805 23:10:14.011727 27171 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260805 23:10:14.011814 27165 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:10:14.012545 27165 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:10:14.018600 27165 catalog_manager.cc:1347] Generated new cluster ID: 1d67c5141cdf41ccb79d3c56e755aa70
I20260805 23:10:14.018678 27165 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:10:14.031715 27165 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:10:14.032987 27165 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:10:14.039727 27165 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 94ae863b48d343ceb1a8d8b53a9666f7: Generated new TSK 0
I20260805 23:10:14.040591 27165 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.92.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.92.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.92.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.193.keytab.
Entry for principal kudu/127.25.92.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.193.keytab.
I20260805 23:10:14.080415 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.92.193:0
--local_ip_for_outbound_sockets=127.25.92.193
--webserver_interface=127.25.92.193
--webserver_port=0
--tserver_master_addrs=127.25.92.254:38411
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.193.keytab
--principal=kudu/127.25.92.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:46793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:10:14.195079 27177 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:10:14.195448 27177 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:10:14.195505 27177 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:10:14.199661 27177 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:10:14.199760 27177 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:10:14.199817 27177 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260805 23:10:14.199855 27177 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.92.193
I20260805 23:10:14.205116 27177 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:46793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.193.keytab
--principal=kudu/127.25.92.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.92.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.92.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.25.92.254:38411
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27177
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.92.193
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:10:14.206172 27177 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:10:14.207352 27177 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:10:14.217530 27182 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:14.217555 27183 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:14.217710 27185 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:10:14.217971 27177 server_base.cc:1034] running on GCE node
I20260805 23:10:14.218461 27177 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:10:14.219398 27177 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:10:14.220811 27177 hybrid_clock.cc:648] HybridClock initialized: now 1785971414220773 us; error 53 us; skew 500 ppm
Aug 05 23:10:14 dist-test-slave-ss4n krb5kdc[27058](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971414, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:10:14.224184 27177 init.cc:377] Logged in from keytab as kudu/127.25.92.193@KRBTEST.COM (short username kudu)
I20260805 23:10:14.225374 27177 webserver.cc:460] Webserver started at http://127.25.92.193:34689/ using document root <none> and password file <none>
I20260805 23:10:14.226002 27177 fs_manager.cc:359] Metadata directory not provided
I20260805 23:10:14.226111 27177 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:10:14.226362 27177 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:10:14.228472 27177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/minicluster-data/ts-0/data/instance:
uuid: "f0e600b6622b486195171f0d4a535e5a"
format_stamp: "Formatted at 2026-08-05 23:10:14 on dist-test-slave-ss4n"
server_key: "92b713ffc6e007450d919db1a8d7af04"
server_key_iv: "cb586495496587eab03763e4b3ad3432"
server_key_version: "encryptionkey@0"
I20260805 23:10:14.229089 27177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/minicluster-data/ts-0/wal/instance:
uuid: "f0e600b6622b486195171f0d4a535e5a"
format_stamp: "Formatted at 2026-08-05 23:10:14 on dist-test-slave-ss4n"
server_key: "92b713ffc6e007450d919db1a8d7af04"
server_key_iv: "cb586495496587eab03763e4b3ad3432"
server_key_version: "encryptionkey@0"
I20260805 23:10:14.233069 27177 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260805 23:10:14.235733 27192 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:14.237202 27177 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260805 23:10:14.237371 27177 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/minicluster-data/ts-0/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/minicluster-data/ts-0/wal
uuid: "f0e600b6622b486195171f0d4a535e5a"
format_stamp: "Formatted at 2026-08-05 23:10:14 on dist-test-slave-ss4n"
server_key: "92b713ffc6e007450d919db1a8d7af04"
server_key_iv: "cb586495496587eab03763e4b3ad3432"
server_key_version: "encryptionkey@0"
I20260805 23:10:14.237541 27177 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:10:14.251847 27177 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:10:14.255435 27177 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:10:14.255702 27177 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:10:14.257215 27177 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:10:14.257294 27177 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:14.257373 27177 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:10:14.257418 27177 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 05 23:10:14 dist-test-slave-ss4n krb5kdc[27058](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971414, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.193@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM
I20260805 23:10:14.268512 27177 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.193:35881
I20260805 23:10:14.268870 27309 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.193:35881 every 8 connection(s)
I20260805 23:10:14.269618 27177 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/minicluster-data/ts-0/data/info.pb
I20260805 23:10:14.272126 27154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:14.258463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.193:41125 (local address 127.25.92.254:38411)
0805 23:10:14.258651 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:14.258656 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:14.259582 (+ 926us) server_negotiation.cc:407] Connection header received
0805 23:10:14.260835 (+ 1253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:14.260839 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:14.260903 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:14.261023 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:14.262823 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:14.263761 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:14.265108 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:14.265362 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:14.265480 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:14.268120 (+ 2640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:14.268126 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:14.268128 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:14.268159 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:14.270084 (+ 1925us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:14.270693 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:14.270696 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:14.270697 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:14.270779 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:14.271093 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:14.271095 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:14.271096 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:14.271279 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:14.271359 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:14.271880 (+ 521us) server_negotiation.cc:299] Negotiation successful
0805 23:10:14.271995 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260805 23:10:14.272953 27201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:14.258915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:38411 (local address 127.25.92.193:41125)
0805 23:10:14.259399 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:14.259455 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:14.260617 (+ 1162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:14.261259 (+ 642us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:14.261270 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:14.261836 (+ 566us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:14.262624 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:14.262648 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:14.263891 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:14.263894 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:14.264975 (+ 1081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:14.264986 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:14.265569 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:14.265574 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:14.265762 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:14.266437 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:10:14.266459 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:14.267942 (+ 1483us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:10:14.270233 (+ 2291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:14.270238 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:14.270253 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:14.270588 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:14.270863 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:14.270866 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:14.270867 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:14.271002 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:14.271410 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:10:14.271415 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:10:14.271660 (+ 245us) client_negotiation.cc:769] Sending connection context
0805 23:10:14.271831 (+ 171us) client_negotiation.cc:240] Negotiation successful
0805 23:10:14.272087 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":135,"threads_started":1}
I20260805 23:10:14.274394 27310 heartbeater.cc:347] Connected to a master server at 127.25.92.254:38411
I20260805 23:10:14.274624 27310 heartbeater.cc:464] Registering TS with master...
I20260805 23:10:14.275247 27310 heartbeater.cc:511] Master 127.25.92.254:38411 requested a full tablet report, sending...
I20260805 23:10:14.276945 27103 ts_manager.cc:194] Registered new tserver with Master: f0e600b6622b486195171f0d4a535e5a (127.25.92.193:35881)
I20260805 23:10:14.278335 27103 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.92.193@KRBTEST.COM'} at 127.25.92.193:41125
I20260805 23:10:14.278532 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 27177
I20260805 23:10:14.278697 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/minicluster-data/ts-0/wal/instance
I20260805 23:10:14.278996 25971 external_mini_cluster.cc:1428] Setting key b89d39d5ecca2d6f27bbb79b82fd852e
I20260805 23:10:14.285444 27154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:14.277051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.193:49967 (local address 127.25.92.254:38411)
0805 23:10:14.277636 (+ 585us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:14.277641 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:14.277655 (+ 14us) server_negotiation.cc:407] Connection header received
0805 23:10:14.277690 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:14.277693 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:14.277734 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:14.277812 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:14.278327 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:14.279043 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:14.280201 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:14.280558 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:14.280667 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:14.282075 (+ 1408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:14.282082 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:14.282088 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:14.282120 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:14.283735 (+ 1615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:14.284244 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:14.284247 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:14.284248 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:14.284317 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:14.284628 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:14.284631 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:14.284632 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:14.284797 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:14.284873 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:14.285219 (+ 346us) server_negotiation.cc:299] Negotiation successful
0805 23:10:14.285322 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":508}
I20260805 23:10:14.285677 27316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:14.276974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:38411 (local address 127.25.92.193:49967)
0805 23:10:14.277304 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:14.277318 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:14.277402 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:14.277868 (+ 466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:14.277870 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:14.278063 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:14.278166 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:14.278171 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:14.279194 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:14.279198 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:14.280065 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:14.280076 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:14.280798 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:14.280800 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:14.280873 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:14.281446 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:10:14.281457 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:14.281918 (+ 461us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:10:14.283854 (+ 1936us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:14.283861 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:14.283867 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:14.284142 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:14.284410 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:14.284416 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:14.284421 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:14.284525 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:14.284904 (+ 379us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:10:14.284908 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:10:14.285020 (+ 112us) client_negotiation.cc:769] Sending connection context
0805 23:10:14.285158 (+ 138us) client_negotiation.cc:240] Negotiation successful
0805 23:10:14.285335 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":126,"threads_started":1}
WARNING: no policy specified for kudu/127.25.92.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.92.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.92.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.194.keytab.
Entry for principal kudu/127.25.92.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.194.keytab.
I20260805 23:10:14.310352 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.92.194:0
--local_ip_for_outbound_sockets=127.25.92.194
--webserver_interface=127.25.92.194
--webserver_port=0
--tserver_master_addrs=127.25.92.254:38411
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.194.keytab
--principal=kudu/127.25.92.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:46793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:10:14.428141 27319 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:10:14.428471 27319 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:10:14.428557 27319 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:10:14.432741 27319 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:10:14.432847 27319 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:10:14.432928 27319 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260805 23:10:14.433014 27319 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.92.194
I20260805 23:10:14.438630 27319 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:46793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.194.keytab
--principal=kudu/127.25.92.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.92.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.92.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.25.92.254:38411
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27319
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.92.194
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:10:14.439774 27319 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:10:14.440845 27319 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:10:14.451108 27327 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:10:14.451210 27319 server_base.cc:1034] running on GCE node
W20260805 23:10:14.451092 27324 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:14.451093 27325 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:10:14.451884 27319 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:10:14.452572 27319 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:10:14.453774 27319 hybrid_clock.cc:648] HybridClock initialized: now 1785971414453757 us; error 28 us; skew 500 ppm
Aug 05 23:10:14 dist-test-slave-ss4n krb5kdc[27058](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971414, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:10:14.456890 27319 init.cc:377] Logged in from keytab as kudu/127.25.92.194@KRBTEST.COM (short username kudu)
I20260805 23:10:14.458122 27319 webserver.cc:460] Webserver started at http://127.25.92.194:44799/ using document root <none> and password file <none>
I20260805 23:10:14.458845 27319 fs_manager.cc:359] Metadata directory not provided
I20260805 23:10:14.458954 27319 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:10:14.459221 27319 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:10:14.461658 27319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/minicluster-data/ts-1/data/instance:
uuid: "be09a9ef751642739e11c69836526ab5"
format_stamp: "Formatted at 2026-08-05 23:10:14 on dist-test-slave-ss4n"
server_key: "cef766336152c91a4d09c9ea61199071"
server_key_iv: "cb807363846a6dd444a6e6f474d839ea"
server_key_version: "encryptionkey@0"
I20260805 23:10:14.462368 27319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/minicluster-data/ts-1/wal/instance:
uuid: "be09a9ef751642739e11c69836526ab5"
format_stamp: "Formatted at 2026-08-05 23:10:14 on dist-test-slave-ss4n"
server_key: "cef766336152c91a4d09c9ea61199071"
server_key_iv: "cb807363846a6dd444a6e6f474d839ea"
server_key_version: "encryptionkey@0"
I20260805 23:10:14.466868 27319 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260805 23:10:14.469986 27334 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:14.471417 27319 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260805 23:10:14.471546 27319 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/minicluster-data/ts-1/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/minicluster-data/ts-1/wal
uuid: "be09a9ef751642739e11c69836526ab5"
format_stamp: "Formatted at 2026-08-05 23:10:14 on dist-test-slave-ss4n"
server_key: "cef766336152c91a4d09c9ea61199071"
server_key_iv: "cb807363846a6dd444a6e6f474d839ea"
server_key_version: "encryptionkey@0"
I20260805 23:10:14.471658 27319 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:10:14.492235 27319 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:10:14.495522 27319 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:10:14.495745 27319 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:10:14.497260 27319 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:10:14.497311 27319 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:14.497390 27319 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:10:14.497423 27319 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 05 23:10:14 dist-test-slave-ss4n krb5kdc[27058](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971414, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.194@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM
I20260805 23:10:14.508831 27319 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.194:35145
I20260805 23:10:14.508958 27451 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.194:35145 every 8 connection(s)
I20260805 23:10:14.509904 27319 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/minicluster-data/ts-1/data/info.pb
I20260805 23:10:14.513113 27154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:14.498248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.194:51827 (local address 127.25.92.254:38411)
0805 23:10:14.498386 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:14.498399 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:14.499524 (+ 1125us) server_negotiation.cc:407] Connection header received
0805 23:10:14.500786 (+ 1262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:14.500792 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:14.500863 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:14.500965 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:14.502539 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:14.503454 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:14.504974 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:14.505352 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:14.505483 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:14.508970 (+ 3487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:14.508978 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:14.508983 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:14.509018 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:14.510941 (+ 1923us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:14.511449 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:14.511452 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:14.511453 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:14.511586 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:14.511985 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:14.511988 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:14.511990 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:14.512212 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:14.512314 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:14.512842 (+ 528us) server_negotiation.cc:299] Negotiation successful
0805 23:10:14.512961 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260805 23:10:14.513937 27343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:14.498798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:38411 (local address 127.25.92.194:51827)
0805 23:10:14.499363 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:14.499406 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:14.500486 (+ 1080us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:14.501208 (+ 722us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:14.501220 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:14.501727 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:14.502395 (+ 668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:14.502410 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:14.503604 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:14.503607 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:14.504836 (+ 1229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:14.504845 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:14.505553 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:14.505558 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:14.505760 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:14.506466 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:10:14.506490 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:14.508767 (+ 2277us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:10:14.511071 (+ 2304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:14.511076 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:14.511086 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:14.511352 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:14.511700 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:14.511702 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:14.511704 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:14.511841 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:14.512339 (+ 498us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:10:14.512344 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:10:14.512587 (+ 243us) client_negotiation.cc:769] Sending connection context
0805 23:10:14.512756 (+ 169us) client_negotiation.cc:240] Negotiation successful
0805 23:10:14.513023 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":160,"threads_started":1}
I20260805 23:10:14.515364 27452 heartbeater.cc:347] Connected to a master server at 127.25.92.254:38411
I20260805 23:10:14.515544 27452 heartbeater.cc:464] Registering TS with master...
I20260805 23:10:14.516045 27452 heartbeater.cc:511] Master 127.25.92.254:38411 requested a full tablet report, sending...
I20260805 23:10:14.517220 27104 ts_manager.cc:194] Registered new tserver with Master: be09a9ef751642739e11c69836526ab5 (127.25.92.194:35145)
I20260805 23:10:14.518085 27104 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.92.194@KRBTEST.COM'} at 127.25.92.194:51827
I20260805 23:10:14.518435 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 27319
I20260805 23:10:14.518546 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/minicluster-data/ts-1/wal/instance
I20260805 23:10:14.518862 25971 external_mini_cluster.cc:1428] Setting key e4dd4c194b78e3306723e3c04b33ba5b
I20260805 23:10:14.527078 27458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:14.517781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:38411 (local address 127.25.92.194:41175)
0805 23:10:14.518211 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:14.518227 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:14.518340 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:14.518746 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:14.518749 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:14.518979 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:14.519107 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:14.519115 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:14.520207 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:14.520213 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:14.521159 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:14.521173 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:14.521807 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:14.521814 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:14.521948 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:14.522607 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:10:14.522622 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:14.522936 (+ 314us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:10:14.525396 (+ 2460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:14.525398 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:14.525400 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:14.525651 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:14.525960 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:14.525962 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:14.525963 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:14.526037 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:14.526624 (+ 587us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:10:14.526627 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:10:14.526705 (+ 78us) client_negotiation.cc:769] Sending connection context
0805 23:10:14.526791 (+ 86us) client_negotiation.cc:240] Negotiation successful
0805 23:10:14.526913 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":114,"threads_started":1}
I20260805 23:10:14.527273 27154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:14.517862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.194:41175 (local address 127.25.92.254:38411)
0805 23:10:14.518002 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:14.518005 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:14.518292 (+ 287us) server_negotiation.cc:407] Connection header received
0805 23:10:14.518459 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:14.518464 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:14.518524 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:14.518613 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:14.519243 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:14.520085 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:14.521305 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:14.521683 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:14.521780 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:14.523106 (+ 1326us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:14.523111 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:14.523114 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:14.523148 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:14.525281 (+ 2133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:14.525750 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:14.525755 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:14.525759 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:14.525861 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:14.526154 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:14.526161 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:14.526165 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:14.526507 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:14.526630 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:14.526890 (+ 260us) server_negotiation.cc:299] Negotiation successful
0805 23:10:14.527044 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
WARNING: no policy specified for kudu/127.25.92.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.92.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.92.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.195.keytab.
Entry for principal kudu/127.25.92.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.195.keytab.
I20260805 23:10:14.549945 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.92.195:0
--local_ip_for_outbound_sockets=127.25.92.195
--webserver_interface=127.25.92.195
--webserver_port=0
--tserver_master_addrs=127.25.92.254:38411
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.195.keytab
--principal=kudu/127.25.92.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:46793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:10:14.671416 27461 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:10:14.671734 27461 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:10:14.671815 27461 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:10:14.676148 27461 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:10:14.676240 27461 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:10:14.676307 27461 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260805 23:10:14.676394 27461 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.92.195
I20260805 23:10:14.681670 27461 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:46793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.195.keytab
--principal=kudu/127.25.92.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.92.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.92.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.25.92.254:38411
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27461
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.92.195
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:10:14.682798 27461 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:10:14.683854 27461 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:10:14.693707 27468 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:14.693665 27471 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:10:14.693660 27461 server_base.cc:1034] running on GCE node
W20260805 23:10:14.693670 27467 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:10:14.694366 27461 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:10:14.695006 27461 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:10:14.696249 27461 hybrid_clock.cc:648] HybridClock initialized: now 1785971414696204 us; error 53 us; skew 500 ppm
Aug 05 23:10:14 dist-test-slave-ss4n krb5kdc[27058](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971414, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:10:14.699218 27461 init.cc:377] Logged in from keytab as kudu/127.25.92.195@KRBTEST.COM (short username kudu)
I20260805 23:10:14.700160 27461 webserver.cc:460] Webserver started at http://127.25.92.195:36149/ using document root <none> and password file <none>
I20260805 23:10:14.700739 27461 fs_manager.cc:359] Metadata directory not provided
I20260805 23:10:14.700806 27461 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:10:14.701016 27461 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:10:14.703207 27461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/minicluster-data/ts-2/data/instance:
uuid: "3c32af5cab1c4fd48fe522f4a1179c54"
format_stamp: "Formatted at 2026-08-05 23:10:14 on dist-test-slave-ss4n"
server_key: "95f663030aa4403fd6ca880ea949da32"
server_key_iv: "ab3235da4e9abf3abcf7d846ff2d45cd"
server_key_version: "encryptionkey@0"
I20260805 23:10:14.703791 27461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/minicluster-data/ts-2/wal/instance:
uuid: "3c32af5cab1c4fd48fe522f4a1179c54"
format_stamp: "Formatted at 2026-08-05 23:10:14 on dist-test-slave-ss4n"
server_key: "95f663030aa4403fd6ca880ea949da32"
server_key_iv: "ab3235da4e9abf3abcf7d846ff2d45cd"
server_key_version: "encryptionkey@0"
I20260805 23:10:14.707856 27461 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260805 23:10:14.710623 27478 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:14.712038 27461 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:10:14.712206 27461 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/minicluster-data/ts-2/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/minicluster-data/ts-2/wal
uuid: "3c32af5cab1c4fd48fe522f4a1179c54"
format_stamp: "Formatted at 2026-08-05 23:10:14 on dist-test-slave-ss4n"
server_key: "95f663030aa4403fd6ca880ea949da32"
server_key_iv: "ab3235da4e9abf3abcf7d846ff2d45cd"
server_key_version: "encryptionkey@0"
I20260805 23:10:14.712337 27461 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:10:14.736708 27461 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:10:14.740408 27461 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:10:14.740690 27461 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:10:14.742573 27461 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:10:14.742671 27461 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:14.742750 27461 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:10:14.742790 27461 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 05 23:10:14 dist-test-slave-ss4n krb5kdc[27058](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971414, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.195@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM
I20260805 23:10:14.755949 27461 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.195:42455
I20260805 23:10:14.756029 27595 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.195:42455 every 8 connection(s)
I20260805 23:10:14.757299 27461 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/minicluster-data/ts-2/data/info.pb
I20260805 23:10:14.757603 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 27461
I20260805 23:10:14.757730 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/minicluster-data/ts-2/wal/instance
I20260805 23:10:14.758198 25971 external_mini_cluster.cc:1428] Setting key bfdc4929208e6a15fce0a2248363f018
I20260805 23:10:14.760365 27154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:14.743652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.195:42357 (local address 127.25.92.254:38411)
0805 23:10:14.743806 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:14.743810 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:14.744934 (+ 1124us) server_negotiation.cc:407] Connection header received
0805 23:10:14.746156 (+ 1222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:14.746162 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:14.746238 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:14.746362 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:14.748091 (+ 1729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:14.749019 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:14.751285 (+ 2266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:14.751713 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:14.751840 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:14.755563 (+ 3723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:14.755567 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:14.755573 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:14.755617 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:14.757652 (+ 2035us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:14.758372 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:14.758376 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:14.758378 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:14.758478 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:14.759052 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:14.759056 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:14.759057 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:14.759320 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:14.759430 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:14.760059 (+ 629us) server_negotiation.cc:299] Negotiation successful
0805 23:10:14.760192 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260805 23:10:14.761364 27487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:14.744226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:38411 (local address 127.25.92.195:42357)
0805 23:10:14.744768 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:14.744811 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:14.745884 (+ 1073us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:14.746629 (+ 745us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:14.746642 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:14.747185 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:14.747909 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:14.747932 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:14.749183 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:14.749188 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:14.751106 (+ 1918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:14.751121 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:14.751929 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:14.751944 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:14.752226 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:14.752970 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:10:14.752997 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:14.755274 (+ 2277us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:10:14.757817 (+ 2543us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:14.757824 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:14.757841 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:14.758206 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:14.758646 (+ 440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:14.758649 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:14.758651 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:14.758861 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:14.759453 (+ 592us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:10:14.759461 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:10:14.759775 (+ 314us) client_negotiation.cc:769] Sending connection context
0805 23:10:14.759936 (+ 161us) client_negotiation.cc:240] Negotiation successful
0805 23:10:14.760348 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":156,"threads_started":1}
I20260805 23:10:14.762645 27596 heartbeater.cc:347] Connected to a master server at 127.25.92.254:38411
I20260805 23:10:14.762809 27596 heartbeater.cc:464] Registering TS with master...
I20260805 23:10:14.763336 27596 heartbeater.cc:511] Master 127.25.92.254:38411 requested a full tablet report, sending...
I20260805 23:10:14.764424 27104 ts_manager.cc:194] Registered new tserver with Master: 3c32af5cab1c4fd48fe522f4a1179c54 (127.25.92.195:42455)
I20260805 23:10:14.765173 27104 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.92.195@KRBTEST.COM'} at 127.25.92.195:42357
I20260805 23:10:14.773012 27154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:14.764954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.195:52167 (local address 127.25.92.254:38411)
0805 23:10:14.765120 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:14.765125 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:14.765301 (+ 176us) server_negotiation.cc:407] Connection header received
0805 23:10:14.765433 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:14.765436 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:14.765573 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:14.765670 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:14.766180 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:14.766992 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:14.767808 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:14.768069 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:14.768158 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:14.769291 (+ 1133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:14.769294 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:14.769296 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:14.769318 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:14.770892 (+ 1574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:14.771541 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:14.771544 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:14.771546 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:14.771685 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:14.772053 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:14.772067 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:14.772069 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:14.772258 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:14.772388 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:14.772711 (+ 323us) server_negotiation.cc:299] Negotiation successful
0805 23:10:14.772876 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260805 23:10:14.773133 25971 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260805 23:10:14.773182 27602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:14.764889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:38411 (local address 127.25.92.195:52167)
0805 23:10:14.765199 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:14.765215 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:14.765317 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:14.765694 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:14.765697 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:14.765938 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:14.766056 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:14.766062 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:14.767143 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:14.767146 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:14.767695 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:14.767703 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:14.768180 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:14.768182 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:14.768260 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:14.768777 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:10:14.768788 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:14.769119 (+ 331us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:10:14.771023 (+ 1904us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:14.771027 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:14.771029 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:14.771334 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:14.771810 (+ 476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:14.771817 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:14.771822 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:14.771936 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:14.772377 (+ 441us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:10:14.772380 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:10:14.772516 (+ 136us) client_negotiation.cc:769] Sending connection context
0805 23:10:14.772652 (+ 136us) client_negotiation.cc:240] Negotiation successful
0805 23:10:14.772891 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219,"mutex_wait_us":1,"thread_start_us":134,"threads_started":1}
Aug 05 23:10:14 dist-test-slave-ss4n krb5kdc[27058](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971414, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 05 23:10:14 dist-test-slave-ss4n krb5kdc[27058](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971414, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM
I20260805 23:10:14.802578 27154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:14.792963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36266 (local address 127.25.92.254:38411)
0805 23:10:14.793130 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:14.793135 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:14.793149 (+ 14us) server_negotiation.cc:407] Connection header received
0805 23:10:14.793196 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:14.793199 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:14.793242 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:14.793319 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:14.794142 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:14.794728 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:14.795587 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:14.795807 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:14.795897 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:14.798648 (+ 2751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:14.798651 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:14.798654 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:14.798683 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:14.800294 (+ 1611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:14.800741 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:14.800744 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:14.800746 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:14.800828 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:14.801166 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:14.801169 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:14.801170 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:14.801364 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:14.801475 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:14.802161 (+ 686us) server_negotiation.cc:299] Negotiation successful
0805 23:10:14.802401 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260805 23:10:14.806161 27104 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36266:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260805 23:10:14.809684 27104 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260805 23:10:14.825860 27615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:14.818868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56000 (local address 127.25.92.195:42455)
0805 23:10:14.820743 (+ 1875us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:14.820751 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:14.820768 (+ 17us) server_negotiation.cc:407] Connection header received
0805 23:10:14.820825 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:14.820828 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:14.821009 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:14.821160 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:10:14.821467 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:14.822663 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:14.824379 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:14.825087 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:14.825213 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:14.825375 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:14.825578 (+ 203us) server_negotiation.cc:299] Negotiation successful
0805 23:10:14.825696 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1793,"thread_start_us":168,"threads_started":1}
I20260805 23:10:14.826334 27616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:14.818694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.195:42455 (local address 127.0.0.1:56000)
0805 23:10:14.820025 (+ 1331us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:14.820095 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:14.820223 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:14.821182 (+ 959us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:14.821186 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:14.821212 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:10:14.821327 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:14.821334 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:14.822798 (+ 1464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:14.822801 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:14.824226 (+ 1425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:14.824237 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:14.825218 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:14.825222 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:14.825330 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:14.825375 (+ 45us) client_negotiation.cc:769] Sending connection context
0805 23:10:14.826033 (+ 658us) client_negotiation.cc:240] Negotiation successful
0805 23:10:14.826185 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1139,"mutex_wait_us":163,"thread_start_us":77,"threads_started":1}
I20260805 23:10:14.826385 27617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:14.819509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58938 (local address 127.25.92.194:35145)
0805 23:10:14.819850 (+ 341us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:14.819863 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:14.820573 (+ 710us) server_negotiation.cc:407] Connection header received
0805 23:10:14.820655 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:14.820659 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:14.820828 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:14.821000 (+ 172us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:10:14.821325 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:14.822643 (+ 1318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:14.824311 (+ 1668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:14.825084 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:14.825198 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:14.825372 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:14.825772 (+ 400us) server_negotiation.cc:299] Negotiation successful
0805 23:10:14.825882 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":228,"thread_start_us":99,"threads_started":1}
I20260805 23:10:14.826540 27613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:14.819096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.193:35881 (local address 127.0.0.1:55162)
0805 23:10:14.821382 (+ 2286us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:14.821393 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:14.821534 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:14.822013 (+ 479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:14.822016 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:14.822035 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:10:14.822120 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:14.822126 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:14.823569 (+ 1443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:14.823573 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:14.824686 (+ 1113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:14.824696 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:14.826105 (+ 1409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:14.826109 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:14.826199 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:14.826227 (+ 28us) client_negotiation.cc:769] Sending connection context
0805 23:10:14.826301 (+ 74us) client_negotiation.cc:240] Negotiation successful
0805 23:10:14.826374 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2220,"thread_start_us":101,"threads_started":1}
I20260805 23:10:14.826767 27614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:14.818928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.194:35145 (local address 127.0.0.1:58938)
0805 23:10:14.820410 (+ 1482us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:14.820419 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:14.820486 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:14.821007 (+ 521us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:14.821014 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:14.821037 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:10:14.821196 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:14.821204 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:14.822791 (+ 1587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:14.822795 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:14.824156 (+ 1361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:14.824167 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:14.825447 (+ 1280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:14.825451 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:14.825582 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:14.825609 (+ 27us) client_negotiation.cc:769] Sending connection context
0805 23:10:14.826536 (+ 927us) client_negotiation.cc:240] Negotiation successful
0805 23:10:14.826630 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1432,"mutex_wait_us":62,"thread_start_us":117,"threads_started":1}
I20260805 23:10:14.827180 27618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:14.819526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55162 (local address 127.25.92.193:35881)
0805 23:10:14.820036 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:14.820043 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:14.821525 (+ 1482us) server_negotiation.cc:407] Connection header received
0805 23:10:14.821655 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:14.821661 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:14.821852 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:14.822034 (+ 182us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:10:14.822231 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:14.823429 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:14.824825 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:14.825737 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:14.825847 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:14.825991 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:14.826939 (+ 948us) server_negotiation.cc:299] Negotiation successful
0805 23:10:14.827061 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":406,"thread_start_us":89,"threads_started":1}
I20260805 23:10:14.829977 27243 tablet_service.cc:1467] Processing CreateTablet for tablet f78953d3dfe14d59bb45198d32008e81 (DEFAULT_TABLE table=test-table [id=71bc7fd26ad04e49ab4079c4f507dedb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:10:14.830051 27385 tablet_service.cc:1467] Processing CreateTablet for tablet f78953d3dfe14d59bb45198d32008e81 (DEFAULT_TABLE table=test-table [id=71bc7fd26ad04e49ab4079c4f507dedb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:10:14.830050 27528 tablet_service.cc:1467] Processing CreateTablet for tablet f78953d3dfe14d59bb45198d32008e81 (DEFAULT_TABLE table=test-table [id=71bc7fd26ad04e49ab4079c4f507dedb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:10:14.831069 27243 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f78953d3dfe14d59bb45198d32008e81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:10:14.831148 27385 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f78953d3dfe14d59bb45198d32008e81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:10:14.831152 27528 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f78953d3dfe14d59bb45198d32008e81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:10:14.837647 27619 tablet_bootstrap.cc:492] T f78953d3dfe14d59bb45198d32008e81 P f0e600b6622b486195171f0d4a535e5a: Bootstrap starting.
I20260805 23:10:14.839915 27621 tablet_bootstrap.cc:492] T f78953d3dfe14d59bb45198d32008e81 P 3c32af5cab1c4fd48fe522f4a1179c54: Bootstrap starting.
I20260805 23:10:14.839990 27619 tablet_bootstrap.cc:654] T f78953d3dfe14d59bb45198d32008e81 P f0e600b6622b486195171f0d4a535e5a: Neither blocks nor log segments found. Creating new log.
I20260805 23:10:14.841632 27620 tablet_bootstrap.cc:492] T f78953d3dfe14d59bb45198d32008e81 P be09a9ef751642739e11c69836526ab5: Bootstrap starting.
I20260805 23:10:14.841786 27619 log.cc:824] T f78953d3dfe14d59bb45198d32008e81 P f0e600b6622b486195171f0d4a535e5a: Log is configured to *not* fsync() on all Append() calls
I20260805 23:10:14.843272 27621 tablet_bootstrap.cc:654] T f78953d3dfe14d59bb45198d32008e81 P 3c32af5cab1c4fd48fe522f4a1179c54: Neither blocks nor log segments found. Creating new log.
I20260805 23:10:14.843802 27620 tablet_bootstrap.cc:654] T f78953d3dfe14d59bb45198d32008e81 P be09a9ef751642739e11c69836526ab5: Neither blocks nor log segments found. Creating new log.
I20260805 23:10:14.844146 27621 log.cc:824] T f78953d3dfe14d59bb45198d32008e81 P 3c32af5cab1c4fd48fe522f4a1179c54: Log is configured to *not* fsync() on all Append() calls
I20260805 23:10:14.844606 27620 log.cc:824] T f78953d3dfe14d59bb45198d32008e81 P be09a9ef751642739e11c69836526ab5: Log is configured to *not* fsync() on all Append() calls
I20260805 23:10:14.846065 27619 tablet_bootstrap.cc:492] T f78953d3dfe14d59bb45198d32008e81 P f0e600b6622b486195171f0d4a535e5a: No bootstrap required, opened a new log
I20260805 23:10:14.846277 27619 ts_tablet_manager.cc:1397] T f78953d3dfe14d59bb45198d32008e81 P f0e600b6622b486195171f0d4a535e5a: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.000s
I20260805 23:10:14.846400 27621 tablet_bootstrap.cc:492] T f78953d3dfe14d59bb45198d32008e81 P 3c32af5cab1c4fd48fe522f4a1179c54: No bootstrap required, opened a new log
I20260805 23:10:14.846590 27621 ts_tablet_manager.cc:1397] T f78953d3dfe14d59bb45198d32008e81 P 3c32af5cab1c4fd48fe522f4a1179c54: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260805 23:10:14.846674 27620 tablet_bootstrap.cc:492] T f78953d3dfe14d59bb45198d32008e81 P be09a9ef751642739e11c69836526ab5: No bootstrap required, opened a new log
I20260805 23:10:14.846859 27620 ts_tablet_manager.cc:1397] T f78953d3dfe14d59bb45198d32008e81 P be09a9ef751642739e11c69836526ab5: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260805 23:10:14.850292 27620 raft_consensus.cc:348] T f78953d3dfe14d59bb45198d32008e81 P be09a9ef751642739e11c69836526ab5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c32af5cab1c4fd48fe522f4a1179c54" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 42455 } } peers { permanent_uuid: "be09a9ef751642739e11c69836526ab5" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 35145 } } peers { permanent_uuid: "f0e600b6622b486195171f0d4a535e5a" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 35881 } }
I20260805 23:10:14.850553 27620 raft_consensus.cc:374] T f78953d3dfe14d59bb45198d32008e81 P be09a9ef751642739e11c69836526ab5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:10:14.850607 27620 raft_consensus.cc:729] T f78953d3dfe14d59bb45198d32008e81 P be09a9ef751642739e11c69836526ab5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be09a9ef751642739e11c69836526ab5, State: Initialized, Role: FOLLOWER
I20260805 23:10:14.850677 27621 raft_consensus.cc:348] T f78953d3dfe14d59bb45198d32008e81 P 3c32af5cab1c4fd48fe522f4a1179c54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c32af5cab1c4fd48fe522f4a1179c54" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 42455 } } peers { permanent_uuid: "be09a9ef751642739e11c69836526ab5" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 35145 } } peers { permanent_uuid: "f0e600b6622b486195171f0d4a535e5a" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 35881 } }
I20260805 23:10:14.850925 27621 raft_consensus.cc:374] T f78953d3dfe14d59bb45198d32008e81 P 3c32af5cab1c4fd48fe522f4a1179c54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:10:14.850984 27621 raft_consensus.cc:729] T f78953d3dfe14d59bb45198d32008e81 P 3c32af5cab1c4fd48fe522f4a1179c54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c32af5cab1c4fd48fe522f4a1179c54, State: Initialized, Role: FOLLOWER
I20260805 23:10:14.851102 27620 consensus_queue.cc:260] T f78953d3dfe14d59bb45198d32008e81 P be09a9ef751642739e11c69836526ab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c32af5cab1c4fd48fe522f4a1179c54" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 42455 } } peers { permanent_uuid: "be09a9ef751642739e11c69836526ab5" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 35145 } } peers { permanent_uuid: "f0e600b6622b486195171f0d4a535e5a" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 35881 } }
I20260805 23:10:14.851263 27619 raft_consensus.cc:348] T f78953d3dfe14d59bb45198d32008e81 P f0e600b6622b486195171f0d4a535e5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c32af5cab1c4fd48fe522f4a1179c54" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 42455 } } peers { permanent_uuid: "be09a9ef751642739e11c69836526ab5" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 35145 } } peers { permanent_uuid: "f0e600b6622b486195171f0d4a535e5a" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 35881 } }
I20260805 23:10:14.851377 27621 consensus_queue.cc:260] T f78953d3dfe14d59bb45198d32008e81 P 3c32af5cab1c4fd48fe522f4a1179c54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c32af5cab1c4fd48fe522f4a1179c54" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 42455 } } peers { permanent_uuid: "be09a9ef751642739e11c69836526ab5" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 35145 } } peers { permanent_uuid: "f0e600b6622b486195171f0d4a535e5a" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 35881 } }
I20260805 23:10:14.851501 27619 raft_consensus.cc:374] T f78953d3dfe14d59bb45198d32008e81 P f0e600b6622b486195171f0d4a535e5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:10:14.851562 27619 raft_consensus.cc:729] T f78953d3dfe14d59bb45198d32008e81 P f0e600b6622b486195171f0d4a535e5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0e600b6622b486195171f0d4a535e5a, State: Initialized, Role: FOLLOWER
I20260805 23:10:14.852020 27619 consensus_queue.cc:260] T f78953d3dfe14d59bb45198d32008e81 P f0e600b6622b486195171f0d4a535e5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c32af5cab1c4fd48fe522f4a1179c54" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 42455 } } peers { permanent_uuid: "be09a9ef751642739e11c69836526ab5" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 35145 } } peers { permanent_uuid: "f0e600b6622b486195171f0d4a535e5a" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 35881 } }
I20260805 23:10:14.852106 27452 heartbeater.cc:503] Master 127.25.92.254:38411 was elected leader, sending a full tablet report...
I20260805 23:10:14.852267 27620 ts_tablet_manager.cc:1428] T f78953d3dfe14d59bb45198d32008e81 P be09a9ef751642739e11c69836526ab5: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260805 23:10:14.852334 27621 ts_tablet_manager.cc:1428] T f78953d3dfe14d59bb45198d32008e81 P 3c32af5cab1c4fd48fe522f4a1179c54: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260805 23:10:14.852692 27596 heartbeater.cc:503] Master 127.25.92.254:38411 was elected leader, sending a full tablet report...
I20260805 23:10:14.852835 27619 ts_tablet_manager.cc:1428] T f78953d3dfe14d59bb45198d32008e81 P f0e600b6622b486195171f0d4a535e5a: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260805 23:10:14.853142 27310 heartbeater.cc:503] Master 127.25.92.254:38411 was elected leader, sending a full tablet report...
I20260805 23:10:14.888965 27626 raft_consensus.cc:482] T f78953d3dfe14d59bb45198d32008e81 P 3c32af5cab1c4fd48fe522f4a1179c54 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 23:10:14.889250 27626 raft_consensus.cc:504] T f78953d3dfe14d59bb45198d32008e81 P 3c32af5cab1c4fd48fe522f4a1179c54 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c32af5cab1c4fd48fe522f4a1179c54" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 42455 } } peers { permanent_uuid: "be09a9ef751642739e11c69836526ab5" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 35145 } } peers { permanent_uuid: "f0e600b6622b486195171f0d4a535e5a" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 35881 } }
I20260805 23:10:14.891080 27626 leader_election.cc:288] T f78953d3dfe14d59bb45198d32008e81 P 3c32af5cab1c4fd48fe522f4a1179c54 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be09a9ef751642739e11c69836526ab5 (127.25.92.194:35145), f0e600b6622b486195171f0d4a535e5a (127.25.92.193:35881)
I20260805 23:10:14.896792 27487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:14.891335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.193:35881 (local address 127.25.92.195:44023)
0805 23:10:14.891494 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:14.891515 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:14.891638 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:14.892217 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:14.892220 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:14.892241 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:10:14.892417 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:14.892424 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:14.893370 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:14.893376 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:14.894944 (+ 1568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:14.894964 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:14.896276 (+ 1312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:14.896284 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:14.896428 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:14.896469 (+ 41us) client_negotiation.cc:769] Sending connection context
0805 23:10:14.896551 (+ 82us) client_negotiation.cc:240] Negotiation successful
0805 23:10:14.896636 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46,"spinlock_wait_cycles":18816}
I20260805 23:10:14.897238 27618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:14.891758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.195:44023 (local address 127.25.92.193:35881)
0805 23:10:14.891950 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:14.891955 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:14.891974 (+ 19us) server_negotiation.cc:407] Connection header received
0805 23:10:14.892035 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:14.892039 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:14.892111 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:14.892203 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:10:14.892546 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:14.893240 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:14.895092 (+ 1852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:14.896004 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:14.896135 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:14.896206 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:14.896992 (+ 786us) server_negotiation.cc:299] Negotiation successful
0805 23:10:14.897109 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260805 23:10:14.897920 27261 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f78953d3dfe14d59bb45198d32008e81" candidate_uuid: "3c32af5cab1c4fd48fe522f4a1179c54" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0e600b6622b486195171f0d4a535e5a" is_pre_election: true
I20260805 23:10:14.898101 27628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:14.891335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.194:35145 (local address 127.25.92.195:56947)
0805 23:10:14.892154 (+ 819us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:14.892173 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:14.892297 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:14.892780 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:14.892786 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:14.892813 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:10:14.893743 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:14.893758 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:14.894882 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:14.894887 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:14.896349 (+ 1462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:14.896364 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:14.897650 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:14.897653 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:14.897759 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:14.897783 (+ 24us) client_negotiation.cc:769] Sending connection context
0805 23:10:14.897858 (+ 75us) client_negotiation.cc:240] Negotiation successful
0805 23:10:14.897933 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":727,"mutex_wait_us":290,"thread_start_us":147,"threads_started":1}
I20260805 23:10:14.898207 27617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:14.891860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.195:56947 (local address 127.25.92.194:35145)
0805 23:10:14.892042 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:14.892047 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:14.892556 (+ 509us) server_negotiation.cc:407] Connection header received
0805 23:10:14.892618 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:14.892622 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:14.892674 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:14.892753 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:10:14.893906 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:14.894755 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:14.896533 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:14.897271 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:14.897395 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:14.897461 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:14.897920 (+ 459us) server_negotiation.cc:299] Negotiation successful
0805 23:10:14.898026 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260805 23:10:14.898329 27261 raft_consensus.cc:2454] T f78953d3dfe14d59bb45198d32008e81 P f0e600b6622b486195171f0d4a535e5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c32af5cab1c4fd48fe522f4a1179c54 in term 0.
I20260805 23:10:14.898910 27480 leader_election.cc:302] T f78953d3dfe14d59bb45198d32008e81 P 3c32af5cab1c4fd48fe522f4a1179c54 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c32af5cab1c4fd48fe522f4a1179c54, f0e600b6622b486195171f0d4a535e5a; no voters:
I20260805 23:10:14.899199 27405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f78953d3dfe14d59bb45198d32008e81" candidate_uuid: "3c32af5cab1c4fd48fe522f4a1179c54" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be09a9ef751642739e11c69836526ab5" is_pre_election: true
I20260805 23:10:14.899259 27626 raft_consensus.cc:2781] T f78953d3dfe14d59bb45198d32008e81 P 3c32af5cab1c4fd48fe522f4a1179c54 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260805 23:10:14.899380 27626 raft_consensus.cc:482] T f78953d3dfe14d59bb45198d32008e81 P 3c32af5cab1c4fd48fe522f4a1179c54 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260805 23:10:14.899446 27626 raft_consensus.cc:3037] T f78953d3dfe14d59bb45198d32008e81 P 3c32af5cab1c4fd48fe522f4a1179c54 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:10:14.899536 27405 raft_consensus.cc:2454] T f78953d3dfe14d59bb45198d32008e81 P be09a9ef751642739e11c69836526ab5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c32af5cab1c4fd48fe522f4a1179c54 in term 0.
I20260805 23:10:14.900977 27626 raft_consensus.cc:504] T f78953d3dfe14d59bb45198d32008e81 P 3c32af5cab1c4fd48fe522f4a1179c54 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c32af5cab1c4fd48fe522f4a1179c54" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 42455 } } peers { permanent_uuid: "be09a9ef751642739e11c69836526ab5" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 35145 } } peers { permanent_uuid: "f0e600b6622b486195171f0d4a535e5a" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 35881 } }
I20260805 23:10:14.901449 27626 leader_election.cc:288] T f78953d3dfe14d59bb45198d32008e81 P 3c32af5cab1c4fd48fe522f4a1179c54 [CANDIDATE]: Term 1 election: Requested vote from peers be09a9ef751642739e11c69836526ab5 (127.25.92.194:35145), f0e600b6622b486195171f0d4a535e5a (127.25.92.193:35881)
I20260805 23:10:14.901823 27405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f78953d3dfe14d59bb45198d32008e81" candidate_uuid: "3c32af5cab1c4fd48fe522f4a1179c54" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be09a9ef751642739e11c69836526ab5"
I20260805 23:10:14.901984 27405 raft_consensus.cc:3037] T f78953d3dfe14d59bb45198d32008e81 P be09a9ef751642739e11c69836526ab5 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:10:14.902022 27261 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f78953d3dfe14d59bb45198d32008e81" candidate_uuid: "3c32af5cab1c4fd48fe522f4a1179c54" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0e600b6622b486195171f0d4a535e5a"
I20260805 23:10:14.902184 27261 raft_consensus.cc:3037] T f78953d3dfe14d59bb45198d32008e81 P f0e600b6622b486195171f0d4a535e5a [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:10:14.903362 27261 raft_consensus.cc:2454] T f78953d3dfe14d59bb45198d32008e81 P f0e600b6622b486195171f0d4a535e5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c32af5cab1c4fd48fe522f4a1179c54 in term 1.
I20260805 23:10:14.903771 27480 leader_election.cc:302] T f78953d3dfe14d59bb45198d32008e81 P 3c32af5cab1c4fd48fe522f4a1179c54 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c32af5cab1c4fd48fe522f4a1179c54, f0e600b6622b486195171f0d4a535e5a; no voters:
I20260805 23:10:14.903777 27405 raft_consensus.cc:2454] T f78953d3dfe14d59bb45198d32008e81 P be09a9ef751642739e11c69836526ab5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c32af5cab1c4fd48fe522f4a1179c54 in term 1.
I20260805 23:10:14.903980 27626 raft_consensus.cc:2781] T f78953d3dfe14d59bb45198d32008e81 P 3c32af5cab1c4fd48fe522f4a1179c54 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:10:14.904222 27626 raft_consensus.cc:686] T f78953d3dfe14d59bb45198d32008e81 P 3c32af5cab1c4fd48fe522f4a1179c54 [term 1 LEADER]: Becoming Leader. State: Replica: 3c32af5cab1c4fd48fe522f4a1179c54, State: Running, Role: LEADER
I20260805 23:10:14.904608 27626 consensus_queue.cc:237] T f78953d3dfe14d59bb45198d32008e81 P 3c32af5cab1c4fd48fe522f4a1179c54 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c32af5cab1c4fd48fe522f4a1179c54" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 42455 } } peers { permanent_uuid: "be09a9ef751642739e11c69836526ab5" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 35145 } } peers { permanent_uuid: "f0e600b6622b486195171f0d4a535e5a" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 35881 } }
I20260805 23:10:14.908764 27102 catalog_manager.cc:5495] T f78953d3dfe14d59bb45198d32008e81 P 3c32af5cab1c4fd48fe522f4a1179c54 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3c32af5cab1c4fd48fe522f4a1179c54 (127.25.92.195). New cstate: current_term: 1 leader_uuid: "3c32af5cab1c4fd48fe522f4a1179c54" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c32af5cab1c4fd48fe522f4a1179c54" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 42455 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "be09a9ef751642739e11c69836526ab5" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 35145 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f0e600b6622b486195171f0d4a535e5a" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 35881 } health_report { overall_health: UNKNOWN } } }
I20260805 23:10:14.935374 27615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:14.930984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56014 (local address 127.25.92.195:42455)
0805 23:10:14.931119 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:14.931124 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:14.931147 (+ 23us) server_negotiation.cc:407] Connection header received
0805 23:10:14.931322 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:14.931324 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:14.931382 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:14.931540 (+ 158us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 23:10:14.931824 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:14.932437 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:14.933408 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:14.933751 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:14.933846 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:14.934309 (+ 463us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 23:10:14.934852 (+ 543us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 23:10:14.934950 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:14.935147 (+ 197us) server_negotiation.cc:299] Negotiation successful
0805 23:10:14.935215 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260805 23:10:14.942353 27261 raft_consensus.cc:1261] T f78953d3dfe14d59bb45198d32008e81 P f0e600b6622b486195171f0d4a535e5a [term 1 FOLLOWER]: Refusing update from remote peer 3c32af5cab1c4fd48fe522f4a1179c54: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:10:14.943198 27629 consensus_queue.cc:1035] T f78953d3dfe14d59bb45198d32008e81 P 3c32af5cab1c4fd48fe522f4a1179c54 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0e600b6622b486195171f0d4a535e5a" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 35881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:10:14.943882 27405 raft_consensus.cc:1261] T f78953d3dfe14d59bb45198d32008e81 P be09a9ef751642739e11c69836526ab5 [term 1 FOLLOWER]: Refusing update from remote peer 3c32af5cab1c4fd48fe522f4a1179c54: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:10:14.944459 27629 consensus_queue.cc:1035] T f78953d3dfe14d59bb45198d32008e81 P 3c32af5cab1c4fd48fe522f4a1179c54 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be09a9ef751642739e11c69836526ab5" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 35145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:10:14.952205 27632 mvcc.cc:205] Tried to move back new op lower bound from 7315338915594645504 to 7315338915452276736. Current Snapshot: MvccSnapshot[applied={T|T < 7315338915594645504}]
I20260805 23:10:14.953318 27634 mvcc.cc:205] Tried to move back new op lower bound from 7315338915594645504 to 7315338915452276736. Current Snapshot: MvccSnapshot[applied={T|T < 7315338915594645504}]
I20260805 23:10:14.953691 27633 mvcc.cc:205] Tried to move back new op lower bound from 7315338915594645504 to 7315338915452276736. Current Snapshot: MvccSnapshot[applied={T|T < 7315338915594645504}]
I20260805 23:10:14.966682 27102 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36266:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:10:14.967023 27102 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36266:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:10:14.970219 27102 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 94ae863b48d343ceb1a8d8b53a9666f7: Sending DeleteTablet for 3 replicas of tablet f78953d3dfe14d59bb45198d32008e81
I20260805 23:10:14.971252 27243 tablet_service.cc:1514] Processing DeleteTablet for tablet f78953d3dfe14d59bb45198d32008e81 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:10:14 UTC) from {username='kudu', principal='kudu/127.25.92.254@KRBTEST.COM'} at 127.0.0.1:55162
I20260805 23:10:14.971252 27528 tablet_service.cc:1514] Processing DeleteTablet for tablet f78953d3dfe14d59bb45198d32008e81 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:10:14 UTC) from {username='kudu', principal='kudu/127.25.92.254@KRBTEST.COM'} at 127.0.0.1:56000
I20260805 23:10:14.971328 27385 tablet_service.cc:1514] Processing DeleteTablet for tablet f78953d3dfe14d59bb45198d32008e81 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:10:14 UTC) from {username='kudu', principal='kudu/127.25.92.254@KRBTEST.COM'} at 127.0.0.1:58938
I20260805 23:10:14.972169 27643 tablet_replica.cc:330] T f78953d3dfe14d59bb45198d32008e81 P f0e600b6622b486195171f0d4a535e5a: stopping tablet replica
I20260805 23:10:14.972365 27642 tablet_replica.cc:330] T f78953d3dfe14d59bb45198d32008e81 P 3c32af5cab1c4fd48fe522f4a1179c54: stopping tablet replica
I20260805 23:10:14.972676 27642 raft_consensus.cc:2229] T f78953d3dfe14d59bb45198d32008e81 P 3c32af5cab1c4fd48fe522f4a1179c54 [term 1 LEADER]: Raft consensus shutting down.
I20260805 23:10:14.972936 27643 raft_consensus.cc:2229] T f78953d3dfe14d59bb45198d32008e81 P f0e600b6622b486195171f0d4a535e5a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 23:10:14.973264 27643 raft_consensus.cc:2258] T f78953d3dfe14d59bb45198d32008e81 P f0e600b6622b486195171f0d4a535e5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:10:14.973232 27102 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36266:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260805 23:10:14.973770 27642 raft_consensus.cc:2258] T f78953d3dfe14d59bb45198d32008e81 P 3c32af5cab1c4fd48fe522f4a1179c54 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260805 23:10:14.973773 27102 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260805 23:10:14.976140 27643 ts_tablet_manager.cc:1905] T f78953d3dfe14d59bb45198d32008e81 P f0e600b6622b486195171f0d4a535e5a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:10:14.977886 27642 ts_tablet_manager.cc:1905] T f78953d3dfe14d59bb45198d32008e81 P 3c32af5cab1c4fd48fe522f4a1179c54: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:10:14.979827 27243 tablet_service.cc:1467] Processing CreateTablet for tablet f80b5a6bf5e0462ab10ef1aec4375874 (DEFAULT_TABLE table=test-table [id=59396ff78e90423482e00d61ec412533]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:10:14.980129 27243 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f80b5a6bf5e0462ab10ef1aec4375874. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:10:14.980194 27385 tablet_service.cc:1467] Processing CreateTablet for tablet f80b5a6bf5e0462ab10ef1aec4375874 (DEFAULT_TABLE table=test-table [id=59396ff78e90423482e00d61ec412533]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:10:14.980484 27385 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f80b5a6bf5e0462ab10ef1aec4375874. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:10:14.981143 27642 ts_tablet_manager.cc:1918] T f78953d3dfe14d59bb45198d32008e81 P 3c32af5cab1c4fd48fe522f4a1179c54: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 23:10:14.981289 27642 log.cc:1196] T f78953d3dfe14d59bb45198d32008e81 P 3c32af5cab1c4fd48fe522f4a1179c54: Deleting WAL directory at /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/minicluster-data/ts-2/wal/wals/f78953d3dfe14d59bb45198d32008e81
I20260805 23:10:14.981818 27642 ts_tablet_manager.cc:1939] T f78953d3dfe14d59bb45198d32008e81 P 3c32af5cab1c4fd48fe522f4a1179c54: Deleting consensus metadata
I20260805 23:10:14.982671 27619 tablet_bootstrap.cc:492] T f80b5a6bf5e0462ab10ef1aec4375874 P f0e600b6622b486195171f0d4a535e5a: Bootstrap starting.
I20260805 23:10:14.983803 27619 tablet_bootstrap.cc:654] T f80b5a6bf5e0462ab10ef1aec4375874 P f0e600b6622b486195171f0d4a535e5a: Neither blocks nor log segments found. Creating new log.
I20260805 23:10:14.983949 27620 tablet_bootstrap.cc:492] T f80b5a6bf5e0462ab10ef1aec4375874 P be09a9ef751642739e11c69836526ab5: Bootstrap starting.
I20260805 23:10:14.984306 27091 catalog_manager.cc:4841] TS 3c32af5cab1c4fd48fe522f4a1179c54 (127.25.92.195:42455): tablet f78953d3dfe14d59bb45198d32008e81 (table test-table [id=71bc7fd26ad04e49ab4079c4f507dedb]) successfully deleted
I20260805 23:10:14.984990 27620 tablet_bootstrap.cc:654] T f80b5a6bf5e0462ab10ef1aec4375874 P be09a9ef751642739e11c69836526ab5: Neither blocks nor log segments found. Creating new log.
I20260805 23:10:14.985091 27619 tablet_bootstrap.cc:492] T f80b5a6bf5e0462ab10ef1aec4375874 P f0e600b6622b486195171f0d4a535e5a: No bootstrap required, opened a new log
I20260805 23:10:14.985174 27619 ts_tablet_manager.cc:1397] T f80b5a6bf5e0462ab10ef1aec4375874 P f0e600b6622b486195171f0d4a535e5a: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260805 23:10:14.986440 27528 tablet_service.cc:1467] Processing CreateTablet for tablet f80b5a6bf5e0462ab10ef1aec4375874 (DEFAULT_TABLE table=test-table [id=59396ff78e90423482e00d61ec412533]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:10:14.986641 27643 ts_tablet_manager.cc:1918] T f78953d3dfe14d59bb45198d32008e81 P f0e600b6622b486195171f0d4a535e5a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 23:10:14.986783 27643 log.cc:1196] T f78953d3dfe14d59bb45198d32008e81 P f0e600b6622b486195171f0d4a535e5a: Deleting WAL directory at /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/minicluster-data/ts-0/wal/wals/f78953d3dfe14d59bb45198d32008e81
I20260805 23:10:14.986821 27528 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f80b5a6bf5e0462ab10ef1aec4375874. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:10:14.987179 27643 ts_tablet_manager.cc:1939] T f78953d3dfe14d59bb45198d32008e81 P f0e600b6622b486195171f0d4a535e5a: Deleting consensus metadata
I20260805 23:10:14.988512 27089 catalog_manager.cc:4841] TS f0e600b6622b486195171f0d4a535e5a (127.25.92.193:35881): tablet f78953d3dfe14d59bb45198d32008e81 (table test-table [id=71bc7fd26ad04e49ab4079c4f507dedb]) successfully deleted
I20260805 23:10:14.988701 27619 raft_consensus.cc:348] T f80b5a6bf5e0462ab10ef1aec4375874 P f0e600b6622b486195171f0d4a535e5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0e600b6622b486195171f0d4a535e5a" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 35881 } } peers { permanent_uuid: "be09a9ef751642739e11c69836526ab5" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 35145 } } peers { permanent_uuid: "3c32af5cab1c4fd48fe522f4a1179c54" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 42455 } }
I20260805 23:10:14.988832 27619 raft_consensus.cc:374] T f80b5a6bf5e0462ab10ef1aec4375874 P f0e600b6622b486195171f0d4a535e5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:10:14.988886 27619 raft_consensus.cc:729] T f80b5a6bf5e0462ab10ef1aec4375874 P f0e600b6622b486195171f0d4a535e5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0e600b6622b486195171f0d4a535e5a, State: Initialized, Role: FOLLOWER
I20260805 23:10:14.989022 27619 consensus_queue.cc:260] T f80b5a6bf5e0462ab10ef1aec4375874 P f0e600b6622b486195171f0d4a535e5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0e600b6622b486195171f0d4a535e5a" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 35881 } } peers { permanent_uuid: "be09a9ef751642739e11c69836526ab5" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 35145 } } peers { permanent_uuid: "3c32af5cab1c4fd48fe522f4a1179c54" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 42455 } }
I20260805 23:10:14.989161 27621 tablet_bootstrap.cc:492] T f80b5a6bf5e0462ab10ef1aec4375874 P 3c32af5cab1c4fd48fe522f4a1179c54: Bootstrap starting.
I20260805 23:10:14.989300 27619 ts_tablet_manager.cc:1428] T f80b5a6bf5e0462ab10ef1aec4375874 P f0e600b6622b486195171f0d4a535e5a: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20260805 23:10:14.990032 27644 tablet_replica.cc:330] T f78953d3dfe14d59bb45198d32008e81 P be09a9ef751642739e11c69836526ab5: stopping tablet replica
I20260805 23:10:14.990329 27621 tablet_bootstrap.cc:654] T f80b5a6bf5e0462ab10ef1aec4375874 P 3c32af5cab1c4fd48fe522f4a1179c54: Neither blocks nor log segments found. Creating new log.
I20260805 23:10:14.990520 27644 raft_consensus.cc:2229] T f78953d3dfe14d59bb45198d32008e81 P be09a9ef751642739e11c69836526ab5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 23:10:14.990911 27644 raft_consensus.cc:2258] T f78953d3dfe14d59bb45198d32008e81 P be09a9ef751642739e11c69836526ab5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:10:14.994375 27621 tablet_bootstrap.cc:492] T f80b5a6bf5e0462ab10ef1aec4375874 P 3c32af5cab1c4fd48fe522f4a1179c54: No bootstrap required, opened a new log
I20260805 23:10:14.994473 27621 ts_tablet_manager.cc:1397] T f80b5a6bf5e0462ab10ef1aec4375874 P 3c32af5cab1c4fd48fe522f4a1179c54: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260805 23:10:14.994974 27621 raft_consensus.cc:348] T f80b5a6bf5e0462ab10ef1aec4375874 P 3c32af5cab1c4fd48fe522f4a1179c54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0e600b6622b486195171f0d4a535e5a" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 35881 } } peers { permanent_uuid: "be09a9ef751642739e11c69836526ab5" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 35145 } } peers { permanent_uuid: "3c32af5cab1c4fd48fe522f4a1179c54" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 42455 } }
I20260805 23:10:14.995101 27621 raft_consensus.cc:374] T f80b5a6bf5e0462ab10ef1aec4375874 P 3c32af5cab1c4fd48fe522f4a1179c54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:10:14.995141 27621 raft_consensus.cc:729] T f80b5a6bf5e0462ab10ef1aec4375874 P 3c32af5cab1c4fd48fe522f4a1179c54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c32af5cab1c4fd48fe522f4a1179c54, State: Initialized, Role: FOLLOWER
I20260805 23:10:14.995296 27644 ts_tablet_manager.cc:1905] T f78953d3dfe14d59bb45198d32008e81 P be09a9ef751642739e11c69836526ab5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:10:14.995277 27621 consensus_queue.cc:260] T f80b5a6bf5e0462ab10ef1aec4375874 P 3c32af5cab1c4fd48fe522f4a1179c54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0e600b6622b486195171f0d4a535e5a" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 35881 } } peers { permanent_uuid: "be09a9ef751642739e11c69836526ab5" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 35145 } } peers { permanent_uuid: "3c32af5cab1c4fd48fe522f4a1179c54" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 42455 } }
I20260805 23:10:14.995572 27621 ts_tablet_manager.cc:1428] T f80b5a6bf5e0462ab10ef1aec4375874 P 3c32af5cab1c4fd48fe522f4a1179c54: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260805 23:10:14.998049 27644 ts_tablet_manager.cc:1918] T f78953d3dfe14d59bb45198d32008e81 P be09a9ef751642739e11c69836526ab5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 23:10:14.998190 27644 log.cc:1196] T f78953d3dfe14d59bb45198d32008e81 P be09a9ef751642739e11c69836526ab5: Deleting WAL directory at /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/minicluster-data/ts-1/wal/wals/f78953d3dfe14d59bb45198d32008e81
I20260805 23:10:14.998499 27644 ts_tablet_manager.cc:1939] T f78953d3dfe14d59bb45198d32008e81 P be09a9ef751642739e11c69836526ab5: Deleting consensus metadata
I20260805 23:10:14.999817 27088 catalog_manager.cc:4841] TS be09a9ef751642739e11c69836526ab5 (127.25.92.194:35145): tablet f78953d3dfe14d59bb45198d32008e81 (table test-table [id=71bc7fd26ad04e49ab4079c4f507dedb]) successfully deleted
I20260805 23:10:15.000360 27620 tablet_bootstrap.cc:492] T f80b5a6bf5e0462ab10ef1aec4375874 P be09a9ef751642739e11c69836526ab5: No bootstrap required, opened a new log
I20260805 23:10:15.000445 27620 ts_tablet_manager.cc:1397] T f80b5a6bf5e0462ab10ef1aec4375874 P be09a9ef751642739e11c69836526ab5: Time spent bootstrapping tablet: real 0.017s user 0.000s sys 0.002s
I20260805 23:10:15.000954 27620 raft_consensus.cc:348] T f80b5a6bf5e0462ab10ef1aec4375874 P be09a9ef751642739e11c69836526ab5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0e600b6622b486195171f0d4a535e5a" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 35881 } } peers { permanent_uuid: "be09a9ef751642739e11c69836526ab5" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 35145 } } peers { permanent_uuid: "3c32af5cab1c4fd48fe522f4a1179c54" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 42455 } }
I20260805 23:10:15.001084 27620 raft_consensus.cc:374] T f80b5a6bf5e0462ab10ef1aec4375874 P be09a9ef751642739e11c69836526ab5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:10:15.001118 27620 raft_consensus.cc:729] T f80b5a6bf5e0462ab10ef1aec4375874 P be09a9ef751642739e11c69836526ab5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be09a9ef751642739e11c69836526ab5, State: Initialized, Role: FOLLOWER
I20260805 23:10:15.001242 27620 consensus_queue.cc:260] T f80b5a6bf5e0462ab10ef1aec4375874 P be09a9ef751642739e11c69836526ab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0e600b6622b486195171f0d4a535e5a" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 35881 } } peers { permanent_uuid: "be09a9ef751642739e11c69836526ab5" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 35145 } } peers { permanent_uuid: "3c32af5cab1c4fd48fe522f4a1179c54" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 42455 } }
I20260805 23:10:15.001910 27620 ts_tablet_manager.cc:1428] T f80b5a6bf5e0462ab10ef1aec4375874 P be09a9ef751642739e11c69836526ab5: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
W20260805 23:10:15.008790 27597 tablet.cc:2369] T f80b5a6bf5e0462ab10ef1aec4375874 P 3c32af5cab1c4fd48fe522f4a1179c54: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260805 23:10:15.011106 27453 tablet.cc:2369] T f80b5a6bf5e0462ab10ef1aec4375874 P be09a9ef751642739e11c69836526ab5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260805 23:10:15.021095 27311 tablet.cc:2369] T f80b5a6bf5e0462ab10ef1aec4375874 P f0e600b6622b486195171f0d4a535e5a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260805 23:10:15.043712 27625 raft_consensus.cc:482] T f80b5a6bf5e0462ab10ef1aec4375874 P be09a9ef751642739e11c69836526ab5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 23:10:15.043896 27625 raft_consensus.cc:504] T f80b5a6bf5e0462ab10ef1aec4375874 P be09a9ef751642739e11c69836526ab5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0e600b6622b486195171f0d4a535e5a" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 35881 } } peers { permanent_uuid: "be09a9ef751642739e11c69836526ab5" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 35145 } } peers { permanent_uuid: "3c32af5cab1c4fd48fe522f4a1179c54" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 42455 } }
I20260805 23:10:15.045231 27625 leader_election.cc:288] T f80b5a6bf5e0462ab10ef1aec4375874 P be09a9ef751642739e11c69836526ab5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f0e600b6622b486195171f0d4a535e5a (127.25.92.193:35881), 3c32af5cab1c4fd48fe522f4a1179c54 (127.25.92.195:42455)
I20260805 23:10:15.050693 27615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:15.045636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.194:37913 (local address 127.25.92.195:42455)
0805 23:10:15.045764 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:15.045769 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:15.046065 (+ 296us) server_negotiation.cc:407] Connection header received
0805 23:10:15.046272 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:15.046276 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:15.046410 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:15.046491 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:10:15.046801 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:15.047520 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:15.049268 (+ 1748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:15.049942 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:15.050083 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:15.050140 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:15.050452 (+ 312us) server_negotiation.cc:299] Negotiation successful
0805 23:10:15.050559 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260805 23:10:15.050699 27648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:15.045642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.195:42455 (local address 127.25.92.194:37913)
0805 23:10:15.045926 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:15.045942 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:15.046085 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:15.046532 (+ 447us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:15.046536 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:15.046556 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:10:15.046678 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:15.046684 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:15.047695 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:15.047699 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:15.049122 (+ 1423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:15.049134 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:15.050099 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:15.050106 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:15.050210 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:15.050318 (+ 108us) client_negotiation.cc:769] Sending connection context
0805 23:10:15.050417 (+ 99us) client_negotiation.cc:240] Negotiation successful
0805 23:10:15.050519 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"mutex_wait_us":56,"thread_start_us":106,"threads_started":1}
I20260805 23:10:15.050745 27618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:15.045700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.194:41659 (local address 127.25.92.193:35881)
0805 23:10:15.045836 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:15.045841 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:15.046382 (+ 541us) server_negotiation.cc:407] Connection header received
0805 23:10:15.046618 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:15.046622 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:15.046677 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:15.046720 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:10:15.047148 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:15.047924 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:15.049453 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:15.050002 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:15.050156 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:15.050217 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:15.050520 (+ 303us) server_negotiation.cc:299] Negotiation successful
0805 23:10:15.050610 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260805 23:10:15.051364 27548 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f80b5a6bf5e0462ab10ef1aec4375874" candidate_uuid: "be09a9ef751642739e11c69836526ab5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c32af5cab1c4fd48fe522f4a1179c54" is_pre_election: true
I20260805 23:10:15.051515 27649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:15.045923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.193:35881 (local address 127.25.92.194:41659)
0805 23:10:15.046283 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:15.046300 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:15.046461 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:15.046891 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:15.046895 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:15.046913 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:10:15.047015 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:15.047022 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:15.048080 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:15.048084 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:15.049326 (+ 1242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:15.049334 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:15.050219 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:15.050223 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:15.050364 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:15.050387 (+ 23us) client_negotiation.cc:769] Sending connection context
0805 23:10:15.051290 (+ 903us) client_negotiation.cc:240] Negotiation successful
0805 23:10:15.051372 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"mutex_wait_us":3,"thread_start_us":108,"threads_started":1}
I20260805 23:10:15.051687 27548 raft_consensus.cc:2454] T f80b5a6bf5e0462ab10ef1aec4375874 P 3c32af5cab1c4fd48fe522f4a1179c54 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be09a9ef751642739e11c69836526ab5 in term 0.
I20260805 23:10:15.052380 27338 leader_election.cc:302] T f80b5a6bf5e0462ab10ef1aec4375874 P be09a9ef751642739e11c69836526ab5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c32af5cab1c4fd48fe522f4a1179c54, be09a9ef751642739e11c69836526ab5; no voters:
I20260805 23:10:15.052697 27625 raft_consensus.cc:2781] T f80b5a6bf5e0462ab10ef1aec4375874 P be09a9ef751642739e11c69836526ab5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260805 23:10:15.052779 27625 raft_consensus.cc:482] T f80b5a6bf5e0462ab10ef1aec4375874 P be09a9ef751642739e11c69836526ab5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260805 23:10:15.052822 27625 raft_consensus.cc:3037] T f80b5a6bf5e0462ab10ef1aec4375874 P be09a9ef751642739e11c69836526ab5 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:10:15.052868 27261 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f80b5a6bf5e0462ab10ef1aec4375874" candidate_uuid: "be09a9ef751642739e11c69836526ab5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0e600b6622b486195171f0d4a535e5a" is_pre_election: true
I20260805 23:10:15.053035 27261 raft_consensus.cc:2454] T f80b5a6bf5e0462ab10ef1aec4375874 P f0e600b6622b486195171f0d4a535e5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be09a9ef751642739e11c69836526ab5 in term 0.
I20260805 23:10:15.054165 27625 raft_consensus.cc:504] T f80b5a6bf5e0462ab10ef1aec4375874 P be09a9ef751642739e11c69836526ab5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0e600b6622b486195171f0d4a535e5a" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 35881 } } peers { permanent_uuid: "be09a9ef751642739e11c69836526ab5" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 35145 } } peers { permanent_uuid: "3c32af5cab1c4fd48fe522f4a1179c54" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 42455 } }
I20260805 23:10:15.054625 27625 leader_election.cc:288] T f80b5a6bf5e0462ab10ef1aec4375874 P be09a9ef751642739e11c69836526ab5 [CANDIDATE]: Term 1 election: Requested vote from peers f0e600b6622b486195171f0d4a535e5a (127.25.92.193:35881), 3c32af5cab1c4fd48fe522f4a1179c54 (127.25.92.195:42455)
I20260805 23:10:15.054975 27261 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f80b5a6bf5e0462ab10ef1aec4375874" candidate_uuid: "be09a9ef751642739e11c69836526ab5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0e600b6622b486195171f0d4a535e5a"
I20260805 23:10:15.055128 27261 raft_consensus.cc:3037] T f80b5a6bf5e0462ab10ef1aec4375874 P f0e600b6622b486195171f0d4a535e5a [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:10:15.055106 27548 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f80b5a6bf5e0462ab10ef1aec4375874" candidate_uuid: "be09a9ef751642739e11c69836526ab5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c32af5cab1c4fd48fe522f4a1179c54"
I20260805 23:10:15.055262 27548 raft_consensus.cc:3037] T f80b5a6bf5e0462ab10ef1aec4375874 P 3c32af5cab1c4fd48fe522f4a1179c54 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:10:15.056299 27548 raft_consensus.cc:2454] T f80b5a6bf5e0462ab10ef1aec4375874 P 3c32af5cab1c4fd48fe522f4a1179c54 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be09a9ef751642739e11c69836526ab5 in term 1.
I20260805 23:10:15.056310 27261 raft_consensus.cc:2454] T f80b5a6bf5e0462ab10ef1aec4375874 P f0e600b6622b486195171f0d4a535e5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be09a9ef751642739e11c69836526ab5 in term 1.
I20260805 23:10:15.056659 27338 leader_election.cc:302] T f80b5a6bf5e0462ab10ef1aec4375874 P be09a9ef751642739e11c69836526ab5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c32af5cab1c4fd48fe522f4a1179c54, be09a9ef751642739e11c69836526ab5; no voters:
I20260805 23:10:15.056932 27625 raft_consensus.cc:2781] T f80b5a6bf5e0462ab10ef1aec4375874 P be09a9ef751642739e11c69836526ab5 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:10:15.057123 27625 raft_consensus.cc:686] T f80b5a6bf5e0462ab10ef1aec4375874 P be09a9ef751642739e11c69836526ab5 [term 1 LEADER]: Becoming Leader. State: Replica: be09a9ef751642739e11c69836526ab5, State: Running, Role: LEADER
I20260805 23:10:15.057389 27625 consensus_queue.cc:237] T f80b5a6bf5e0462ab10ef1aec4375874 P be09a9ef751642739e11c69836526ab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0e600b6622b486195171f0d4a535e5a" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 35881 } } peers { permanent_uuid: "be09a9ef751642739e11c69836526ab5" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 35145 } } peers { permanent_uuid: "3c32af5cab1c4fd48fe522f4a1179c54" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 42455 } }
I20260805 23:10:15.059932 27102 catalog_manager.cc:5495] T f80b5a6bf5e0462ab10ef1aec4375874 P be09a9ef751642739e11c69836526ab5 reported cstate change: term changed from 0 to 1, leader changed from <none> to be09a9ef751642739e11c69836526ab5 (127.25.92.194). New cstate: current_term: 1 leader_uuid: "be09a9ef751642739e11c69836526ab5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0e600b6622b486195171f0d4a535e5a" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 35881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be09a9ef751642739e11c69836526ab5" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 35145 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c32af5cab1c4fd48fe522f4a1179c54" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 42455 } health_report { overall_health: UNKNOWN } } }
Aug 05 23:10:15 dist-test-slave-ss4n krb5kdc[27058](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971413, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.254@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM
I20260805 23:10:15.098047 27154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:15.087806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36270 (local address 127.25.92.254:38411)
0805 23:10:15.087952 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:15.087957 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:15.088421 (+ 464us) server_negotiation.cc:407] Connection header received
0805 23:10:15.088529 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:15.088534 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:15.088612 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:15.088686 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:15.089691 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:15.090331 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:15.091325 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:15.091535 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:15.091650 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:15.094558 (+ 2908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:15.094561 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:15.094563 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:15.094586 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:15.096150 (+ 1564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:15.096702 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:15.096707 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:15.096709 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:15.096792 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:15.097072 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:15.097076 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:15.097078 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:15.097260 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:15.097367 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:15.097701 (+ 334us) server_negotiation.cc:299] Negotiation successful
0805 23:10:15.097911 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":127,"server-negotiator.queue_time_us":57}
I20260805 23:10:15.098047 27657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:15.087927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:38411 (local address 127.0.0.1:36270)
0805 23:10:15.088284 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:15.088301 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:15.088425 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:15.088732 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:15.088735 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:15.089371 (+ 636us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:15.089559 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:15.089568 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:15.090438 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:15.090443 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:15.091134 (+ 691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:15.091143 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:15.091654 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:15.091657 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:15.091774 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:15.092432 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:10:15.092444 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:15.094328 (+ 1884us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:10:15.096315 (+ 1987us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:15.096325 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:15.096331 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:15.096570 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:15.096890 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:15.096893 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:15.096895 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:15.096976 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:15.097367 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:10:15.097374 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:10:15.097509 (+ 135us) client_negotiation.cc:769] Sending connection context
0805 23:10:15.097655 (+ 146us) client_negotiation.cc:240] Negotiation successful
0805 23:10:15.097847 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"mutex_wait_us":26,"thread_start_us":142,"threads_started":1}
I20260805 23:10:15.102286 27102 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.25.92.254@KRBTEST.COM'} at 127.0.0.1:36270:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260805 23:10:15.103045 27102 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260805 23:10:15.108634 27243 tablet_service.cc:1467] Processing CreateTablet for tablet 04d98d10ba814df7ad9a3d00860b3c2b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=7ae11c5d02b847f0b3df695822534c61]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260805 23:10:15.108840 27528 tablet_service.cc:1467] Processing CreateTablet for tablet 04d98d10ba814df7ad9a3d00860b3c2b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=7ae11c5d02b847f0b3df695822534c61]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260805 23:10:15.108894 27385 tablet_service.cc:1467] Processing CreateTablet for tablet 04d98d10ba814df7ad9a3d00860b3c2b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=7ae11c5d02b847f0b3df695822534c61]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260805 23:10:15.109174 27243 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 04d98d10ba814df7ad9a3d00860b3c2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:10:15.109196 27528 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 04d98d10ba814df7ad9a3d00860b3c2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:10:15.109305 27385 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 04d98d10ba814df7ad9a3d00860b3c2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:10:15.112210 27619 tablet_bootstrap.cc:492] T 04d98d10ba814df7ad9a3d00860b3c2b P f0e600b6622b486195171f0d4a535e5a: Bootstrap starting.
I20260805 23:10:15.112639 27620 tablet_bootstrap.cc:492] T 04d98d10ba814df7ad9a3d00860b3c2b P be09a9ef751642739e11c69836526ab5: Bootstrap starting.
I20260805 23:10:15.112988 27619 tablet_bootstrap.cc:654] T 04d98d10ba814df7ad9a3d00860b3c2b P f0e600b6622b486195171f0d4a535e5a: Neither blocks nor log segments found. Creating new log.
I20260805 23:10:15.113636 27620 tablet_bootstrap.cc:654] T 04d98d10ba814df7ad9a3d00860b3c2b P be09a9ef751642739e11c69836526ab5: Neither blocks nor log segments found. Creating new log.
I20260805 23:10:15.115010 27621 tablet_bootstrap.cc:492] T 04d98d10ba814df7ad9a3d00860b3c2b P 3c32af5cab1c4fd48fe522f4a1179c54: Bootstrap starting.
I20260805 23:10:15.115267 27620 tablet_bootstrap.cc:492] T 04d98d10ba814df7ad9a3d00860b3c2b P be09a9ef751642739e11c69836526ab5: No bootstrap required, opened a new log
I20260805 23:10:15.115370 27620 ts_tablet_manager.cc:1397] T 04d98d10ba814df7ad9a3d00860b3c2b P be09a9ef751642739e11c69836526ab5: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260805 23:10:15.115862 27620 raft_consensus.cc:348] T 04d98d10ba814df7ad9a3d00860b3c2b P be09a9ef751642739e11c69836526ab5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0e600b6622b486195171f0d4a535e5a" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 35881 } } peers { permanent_uuid: "be09a9ef751642739e11c69836526ab5" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 35145 } } peers { permanent_uuid: "3c32af5cab1c4fd48fe522f4a1179c54" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 42455 } }
I20260805 23:10:15.116010 27620 raft_consensus.cc:374] T 04d98d10ba814df7ad9a3d00860b3c2b P be09a9ef751642739e11c69836526ab5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:10:15.116047 27620 raft_consensus.cc:729] T 04d98d10ba814df7ad9a3d00860b3c2b P be09a9ef751642739e11c69836526ab5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be09a9ef751642739e11c69836526ab5, State: Initialized, Role: FOLLOWER
I20260805 23:10:15.116151 27619 tablet_bootstrap.cc:492] T 04d98d10ba814df7ad9a3d00860b3c2b P f0e600b6622b486195171f0d4a535e5a: No bootstrap required, opened a new log
I20260805 23:10:15.116252 27619 ts_tablet_manager.cc:1397] T 04d98d10ba814df7ad9a3d00860b3c2b P f0e600b6622b486195171f0d4a535e5a: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260805 23:10:15.116312 27620 consensus_queue.cc:260] T 04d98d10ba814df7ad9a3d00860b3c2b P be09a9ef751642739e11c69836526ab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0e600b6622b486195171f0d4a535e5a" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 35881 } } peers { permanent_uuid: "be09a9ef751642739e11c69836526ab5" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 35145 } } peers { permanent_uuid: "3c32af5cab1c4fd48fe522f4a1179c54" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 42455 } }
I20260805 23:10:15.116469 27621 tablet_bootstrap.cc:654] T 04d98d10ba814df7ad9a3d00860b3c2b P 3c32af5cab1c4fd48fe522f4a1179c54: Neither blocks nor log segments found. Creating new log.
I20260805 23:10:15.116603 27620 ts_tablet_manager.cc:1428] T 04d98d10ba814df7ad9a3d00860b3c2b P be09a9ef751642739e11c69836526ab5: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260805 23:10:15.116750 27619 raft_consensus.cc:348] T 04d98d10ba814df7ad9a3d00860b3c2b P f0e600b6622b486195171f0d4a535e5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0e600b6622b486195171f0d4a535e5a" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 35881 } } peers { permanent_uuid: "be09a9ef751642739e11c69836526ab5" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 35145 } } peers { permanent_uuid: "3c32af5cab1c4fd48fe522f4a1179c54" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 42455 } }
I20260805 23:10:15.116860 27619 raft_consensus.cc:374] T 04d98d10ba814df7ad9a3d00860b3c2b P f0e600b6622b486195171f0d4a535e5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:10:15.116900 27619 raft_consensus.cc:729] T 04d98d10ba814df7ad9a3d00860b3c2b P f0e600b6622b486195171f0d4a535e5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0e600b6622b486195171f0d4a535e5a, State: Initialized, Role: FOLLOWER
I20260805 23:10:15.117027 27619 consensus_queue.cc:260] T 04d98d10ba814df7ad9a3d00860b3c2b P f0e600b6622b486195171f0d4a535e5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0e600b6622b486195171f0d4a535e5a" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 35881 } } peers { permanent_uuid: "be09a9ef751642739e11c69836526ab5" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 35145 } } peers { permanent_uuid: "3c32af5cab1c4fd48fe522f4a1179c54" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 42455 } }
I20260805 23:10:15.117337 27619 ts_tablet_manager.cc:1428] T 04d98d10ba814df7ad9a3d00860b3c2b P f0e600b6622b486195171f0d4a535e5a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260805 23:10:15.117370 27625 tablet_replica.cc:439] T 04d98d10ba814df7ad9a3d00860b3c2b P be09a9ef751642739e11c69836526ab5: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0e600b6622b486195171f0d4a535e5a" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 35881 } } peers { permanent_uuid: "be09a9ef751642739e11c69836526ab5" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 35145 } } peers { permanent_uuid: "3c32af5cab1c4fd48fe522f4a1179c54" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 42455 } } }
I20260805 23:10:15.117543 27625 tablet_replica.cc:442] T 04d98d10ba814df7ad9a3d00860b3c2b P be09a9ef751642739e11c69836526ab5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260805 23:10:15.117947 27627 tablet_replica.cc:439] T 04d98d10ba814df7ad9a3d00860b3c2b P f0e600b6622b486195171f0d4a535e5a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0e600b6622b486195171f0d4a535e5a" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 35881 } } peers { permanent_uuid: "be09a9ef751642739e11c69836526ab5" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 35145 } } peers { permanent_uuid: "3c32af5cab1c4fd48fe522f4a1179c54" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 42455 } } }
I20260805 23:10:15.118135 27627 tablet_replica.cc:442] T 04d98d10ba814df7ad9a3d00860b3c2b P f0e600b6622b486195171f0d4a535e5a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260805 23:10:15.119341 27621 tablet_bootstrap.cc:492] T 04d98d10ba814df7ad9a3d00860b3c2b P 3c32af5cab1c4fd48fe522f4a1179c54: No bootstrap required, opened a new log
I20260805 23:10:15.119433 27621 ts_tablet_manager.cc:1397] T 04d98d10ba814df7ad9a3d00860b3c2b P 3c32af5cab1c4fd48fe522f4a1179c54: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260805 23:10:15.119944 27621 raft_consensus.cc:348] T 04d98d10ba814df7ad9a3d00860b3c2b P 3c32af5cab1c4fd48fe522f4a1179c54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0e600b6622b486195171f0d4a535e5a" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 35881 } } peers { permanent_uuid: "be09a9ef751642739e11c69836526ab5" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 35145 } } peers { permanent_uuid: "3c32af5cab1c4fd48fe522f4a1179c54" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 42455 } }
I20260805 23:10:15.120064 27621 raft_consensus.cc:374] T 04d98d10ba814df7ad9a3d00860b3c2b P 3c32af5cab1c4fd48fe522f4a1179c54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:10:15.120102 27621 raft_consensus.cc:729] T 04d98d10ba814df7ad9a3d00860b3c2b P 3c32af5cab1c4fd48fe522f4a1179c54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c32af5cab1c4fd48fe522f4a1179c54, State: Initialized, Role: FOLLOWER
I20260805 23:10:15.120232 27621 consensus_queue.cc:260] T 04d98d10ba814df7ad9a3d00860b3c2b P 3c32af5cab1c4fd48fe522f4a1179c54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0e600b6622b486195171f0d4a535e5a" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 35881 } } peers { permanent_uuid: "be09a9ef751642739e11c69836526ab5" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 35145 } } peers { permanent_uuid: "3c32af5cab1c4fd48fe522f4a1179c54" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 42455 } }
I20260805 23:10:15.120515 27621 ts_tablet_manager.cc:1428] T 04d98d10ba814df7ad9a3d00860b3c2b P 3c32af5cab1c4fd48fe522f4a1179c54: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260805 23:10:15.120801 27626 tablet_replica.cc:439] T 04d98d10ba814df7ad9a3d00860b3c2b P 3c32af5cab1c4fd48fe522f4a1179c54: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0e600b6622b486195171f0d4a535e5a" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 35881 } } peers { permanent_uuid: "be09a9ef751642739e11c69836526ab5" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 35145 } } peers { permanent_uuid: "3c32af5cab1c4fd48fe522f4a1179c54" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 42455 } } }
I20260805 23:10:15.120935 27626 tablet_replica.cc:442] T 04d98d10ba814df7ad9a3d00860b3c2b P 3c32af5cab1c4fd48fe522f4a1179c54: This TxnStatusTablet replica's current role is: FOLLOWER
I20260805 23:10:15.141618 27625 raft_consensus.cc:482] T 04d98d10ba814df7ad9a3d00860b3c2b P be09a9ef751642739e11c69836526ab5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 23:10:15.141793 27625 raft_consensus.cc:504] T 04d98d10ba814df7ad9a3d00860b3c2b P be09a9ef751642739e11c69836526ab5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0e600b6622b486195171f0d4a535e5a" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 35881 } } peers { permanent_uuid: "be09a9ef751642739e11c69836526ab5" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 35145 } } peers { permanent_uuid: "3c32af5cab1c4fd48fe522f4a1179c54" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 42455 } }
I20260805 23:10:15.142320 27625 leader_election.cc:288] T 04d98d10ba814df7ad9a3d00860b3c2b P be09a9ef751642739e11c69836526ab5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f0e600b6622b486195171f0d4a535e5a (127.25.92.193:35881), 3c32af5cab1c4fd48fe522f4a1179c54 (127.25.92.195:42455)
I20260805 23:10:15.142746 27261 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04d98d10ba814df7ad9a3d00860b3c2b" candidate_uuid: "be09a9ef751642739e11c69836526ab5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0e600b6622b486195171f0d4a535e5a" is_pre_election: true
I20260805 23:10:15.142815 27548 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04d98d10ba814df7ad9a3d00860b3c2b" candidate_uuid: "be09a9ef751642739e11c69836526ab5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c32af5cab1c4fd48fe522f4a1179c54" is_pre_election: true
I20260805 23:10:15.142928 27261 raft_consensus.cc:2454] T 04d98d10ba814df7ad9a3d00860b3c2b P f0e600b6622b486195171f0d4a535e5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be09a9ef751642739e11c69836526ab5 in term 0.
I20260805 23:10:15.142984 27548 raft_consensus.cc:2454] T 04d98d10ba814df7ad9a3d00860b3c2b P 3c32af5cab1c4fd48fe522f4a1179c54 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be09a9ef751642739e11c69836526ab5 in term 0.
I20260805 23:10:15.143290 27336 leader_election.cc:302] T 04d98d10ba814df7ad9a3d00860b3c2b P be09a9ef751642739e11c69836526ab5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be09a9ef751642739e11c69836526ab5, f0e600b6622b486195171f0d4a535e5a; no voters:
I20260805 23:10:15.143546 27625 raft_consensus.cc:2781] T 04d98d10ba814df7ad9a3d00860b3c2b P be09a9ef751642739e11c69836526ab5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260805 23:10:15.143613 27625 raft_consensus.cc:482] T 04d98d10ba814df7ad9a3d00860b3c2b P be09a9ef751642739e11c69836526ab5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260805 23:10:15.143671 27625 raft_consensus.cc:3037] T 04d98d10ba814df7ad9a3d00860b3c2b P be09a9ef751642739e11c69836526ab5 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:10:15.144732 27625 raft_consensus.cc:504] T 04d98d10ba814df7ad9a3d00860b3c2b P be09a9ef751642739e11c69836526ab5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0e600b6622b486195171f0d4a535e5a" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 35881 } } peers { permanent_uuid: "be09a9ef751642739e11c69836526ab5" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 35145 } } peers { permanent_uuid: "3c32af5cab1c4fd48fe522f4a1179c54" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 42455 } }
I20260805 23:10:15.145448 27548 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04d98d10ba814df7ad9a3d00860b3c2b" candidate_uuid: "be09a9ef751642739e11c69836526ab5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c32af5cab1c4fd48fe522f4a1179c54"
I20260805 23:10:15.145546 27261 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04d98d10ba814df7ad9a3d00860b3c2b" candidate_uuid: "be09a9ef751642739e11c69836526ab5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0e600b6622b486195171f0d4a535e5a"
I20260805 23:10:15.145627 27548 raft_consensus.cc:3037] T 04d98d10ba814df7ad9a3d00860b3c2b P 3c32af5cab1c4fd48fe522f4a1179c54 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:10:15.145681 27261 raft_consensus.cc:3037] T 04d98d10ba814df7ad9a3d00860b3c2b P f0e600b6622b486195171f0d4a535e5a [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:10:15.146733 27548 raft_consensus.cc:2454] T 04d98d10ba814df7ad9a3d00860b3c2b P 3c32af5cab1c4fd48fe522f4a1179c54 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be09a9ef751642739e11c69836526ab5 in term 1.
I20260805 23:10:15.146853 27261 raft_consensus.cc:2454] T 04d98d10ba814df7ad9a3d00860b3c2b P f0e600b6622b486195171f0d4a535e5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be09a9ef751642739e11c69836526ab5 in term 1.
I20260805 23:10:15.146929 27625 leader_election.cc:288] T 04d98d10ba814df7ad9a3d00860b3c2b P be09a9ef751642739e11c69836526ab5 [CANDIDATE]: Term 1 election: Requested vote from peers f0e600b6622b486195171f0d4a535e5a (127.25.92.193:35881), 3c32af5cab1c4fd48fe522f4a1179c54 (127.25.92.195:42455)
I20260805 23:10:15.147032 27338 leader_election.cc:302] T 04d98d10ba814df7ad9a3d00860b3c2b P be09a9ef751642739e11c69836526ab5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c32af5cab1c4fd48fe522f4a1179c54, be09a9ef751642739e11c69836526ab5; no voters:
I20260805 23:10:15.147192 27625 raft_consensus.cc:2781] T 04d98d10ba814df7ad9a3d00860b3c2b P be09a9ef751642739e11c69836526ab5 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:10:15.147277 27625 raft_consensus.cc:686] T 04d98d10ba814df7ad9a3d00860b3c2b P be09a9ef751642739e11c69836526ab5 [term 1 LEADER]: Becoming Leader. State: Replica: be09a9ef751642739e11c69836526ab5, State: Running, Role: LEADER
I20260805 23:10:15.147411 27625 consensus_queue.cc:237] T 04d98d10ba814df7ad9a3d00860b3c2b P be09a9ef751642739e11c69836526ab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0e600b6622b486195171f0d4a535e5a" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 35881 } } peers { permanent_uuid: "be09a9ef751642739e11c69836526ab5" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 35145 } } peers { permanent_uuid: "3c32af5cab1c4fd48fe522f4a1179c54" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 42455 } }
I20260805 23:10:15.148214 27650 tablet_replica.cc:439] T 04d98d10ba814df7ad9a3d00860b3c2b P be09a9ef751642739e11c69836526ab5: TxnStatusTablet state changed. Reason: New leader be09a9ef751642739e11c69836526ab5. Latest consensus state: current_term: 1 leader_uuid: "be09a9ef751642739e11c69836526ab5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0e600b6622b486195171f0d4a535e5a" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 35881 } } peers { permanent_uuid: "be09a9ef751642739e11c69836526ab5" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 35145 } } peers { permanent_uuid: "3c32af5cab1c4fd48fe522f4a1179c54" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 42455 } } }
I20260805 23:10:15.148334 27650 tablet_replica.cc:442] T 04d98d10ba814df7ad9a3d00860b3c2b P be09a9ef751642739e11c69836526ab5: This TxnStatusTablet replica's current role is: LEADER
I20260805 23:10:15.148741 27662 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260805 23:10:15.148943 27662 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260805 23:10:15.149922 27102 catalog_manager.cc:5495] T 04d98d10ba814df7ad9a3d00860b3c2b P be09a9ef751642739e11c69836526ab5 reported cstate change: term changed from 0 to 1, leader changed from <none> to be09a9ef751642739e11c69836526ab5 (127.25.92.194). New cstate: current_term: 1 leader_uuid: "be09a9ef751642739e11c69836526ab5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0e600b6622b486195171f0d4a535e5a" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 35881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be09a9ef751642739e11c69836526ab5" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 35145 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c32af5cab1c4fd48fe522f4a1179c54" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 42455 } health_report { overall_health: UNKNOWN } } }
I20260805 23:10:15.173213 27657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:15.168724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.194:35145 (local address 127.0.0.1:58944)
0805 23:10:15.168897 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:15.168913 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:15.169015 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:15.169358 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:15.169360 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:15.169366 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0805 23:10:15.169462 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:15.169465 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:15.170530 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:15.170533 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:15.171284 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:15.171294 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:15.171903 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:15.171906 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:15.171979 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:15.172027 (+ 48us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0805 23:10:15.172928 (+ 901us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0805 23:10:15.172943 (+ 15us) client_negotiation.cc:769] Sending connection context
0805 23:10:15.173004 (+ 61us) client_negotiation.cc:240] Negotiation successful
0805 23:10:15.173086 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260805 23:10:15.173260 27617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:15.168810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58944 (local address 127.25.92.194:35145)
0805 23:10:15.168941 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:15.168946 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:15.168962 (+ 16us) server_negotiation.cc:407] Connection header received
0805 23:10:15.169161 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:15.169164 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:15.169233 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:15.169342 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 23:10:15.169592 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:15.170367 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:15.171405 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:15.171767 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:15.171866 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:15.172226 (+ 360us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 23:10:15.172807 (+ 581us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 23:10:15.172920 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:15.173047 (+ 127us) server_negotiation.cc:299] Negotiation successful
0805 23:10:15.173111 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260805 23:10:15.176470 27261 raft_consensus.cc:1261] T 04d98d10ba814df7ad9a3d00860b3c2b P f0e600b6622b486195171f0d4a535e5a [term 1 FOLLOWER]: Refusing update from remote peer be09a9ef751642739e11c69836526ab5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:10:15.176666 27548 raft_consensus.cc:1261] T 04d98d10ba814df7ad9a3d00860b3c2b P 3c32af5cab1c4fd48fe522f4a1179c54 [term 1 FOLLOWER]: Refusing update from remote peer be09a9ef751642739e11c69836526ab5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:10:15.177045 27650 consensus_queue.cc:1035] T 04d98d10ba814df7ad9a3d00860b3c2b P be09a9ef751642739e11c69836526ab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0e600b6622b486195171f0d4a535e5a" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 35881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:10:15.177299 27625 consensus_queue.cc:1035] T 04d98d10ba814df7ad9a3d00860b3c2b P be09a9ef751642739e11c69836526ab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c32af5cab1c4fd48fe522f4a1179c54" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 42455 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:10:15.179046 27627 tablet_replica.cc:439] T 04d98d10ba814df7ad9a3d00860b3c2b P f0e600b6622b486195171f0d4a535e5a: TxnStatusTablet state changed. Reason: New leader be09a9ef751642739e11c69836526ab5. Latest consensus state: current_term: 1 leader_uuid: "be09a9ef751642739e11c69836526ab5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0e600b6622b486195171f0d4a535e5a" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 35881 } } peers { permanent_uuid: "be09a9ef751642739e11c69836526ab5" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 35145 } } peers { permanent_uuid: "3c32af5cab1c4fd48fe522f4a1179c54" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 42455 } } }
I20260805 23:10:15.179169 27627 tablet_replica.cc:442] T 04d98d10ba814df7ad9a3d00860b3c2b P f0e600b6622b486195171f0d4a535e5a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260805 23:10:15.179097 27626 tablet_replica.cc:439] T 04d98d10ba814df7ad9a3d00860b3c2b P 3c32af5cab1c4fd48fe522f4a1179c54: TxnStatusTablet state changed. Reason: New leader be09a9ef751642739e11c69836526ab5. Latest consensus state: current_term: 1 leader_uuid: "be09a9ef751642739e11c69836526ab5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0e600b6622b486195171f0d4a535e5a" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 35881 } } peers { permanent_uuid: "be09a9ef751642739e11c69836526ab5" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 35145 } } peers { permanent_uuid: "3c32af5cab1c4fd48fe522f4a1179c54" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 42455 } } }
I20260805 23:10:15.179227 27626 tablet_replica.cc:442] T 04d98d10ba814df7ad9a3d00860b3c2b P 3c32af5cab1c4fd48fe522f4a1179c54: This TxnStatusTablet replica's current role is: FOLLOWER
I20260805 23:10:15.180725 27650 tablet_replica.cc:439] T 04d98d10ba814df7ad9a3d00860b3c2b P be09a9ef751642739e11c69836526ab5: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "be09a9ef751642739e11c69836526ab5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0e600b6622b486195171f0d4a535e5a" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 35881 } } peers { permanent_uuid: "be09a9ef751642739e11c69836526ab5" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 35145 } } peers { permanent_uuid: "3c32af5cab1c4fd48fe522f4a1179c54" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 42455 } } }
I20260805 23:10:15.180855 27650 tablet_replica.cc:442] T 04d98d10ba814df7ad9a3d00860b3c2b P be09a9ef751642739e11c69836526ab5: This TxnStatusTablet replica's current role is: LEADER
I20260805 23:10:15.183560 27627 tablet_replica.cc:439] T 04d98d10ba814df7ad9a3d00860b3c2b P f0e600b6622b486195171f0d4a535e5a: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "be09a9ef751642739e11c69836526ab5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0e600b6622b486195171f0d4a535e5a" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 35881 } } peers { permanent_uuid: "be09a9ef751642739e11c69836526ab5" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 35145 } } peers { permanent_uuid: "3c32af5cab1c4fd48fe522f4a1179c54" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 42455 } } }
I20260805 23:10:15.183682 27627 tablet_replica.cc:442] T 04d98d10ba814df7ad9a3d00860b3c2b P f0e600b6622b486195171f0d4a535e5a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260805 23:10:15.185508 27625 tablet_replica.cc:439] T 04d98d10ba814df7ad9a3d00860b3c2b P be09a9ef751642739e11c69836526ab5: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "be09a9ef751642739e11c69836526ab5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0e600b6622b486195171f0d4a535e5a" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 35881 } } peers { permanent_uuid: "be09a9ef751642739e11c69836526ab5" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 35145 } } peers { permanent_uuid: "3c32af5cab1c4fd48fe522f4a1179c54" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 42455 } } }
I20260805 23:10:15.185627 27625 tablet_replica.cc:442] T 04d98d10ba814df7ad9a3d00860b3c2b P be09a9ef751642739e11c69836526ab5: This TxnStatusTablet replica's current role is: LEADER
I20260805 23:10:15.186488 27626 tablet_replica.cc:439] T 04d98d10ba814df7ad9a3d00860b3c2b P 3c32af5cab1c4fd48fe522f4a1179c54: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "be09a9ef751642739e11c69836526ab5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0e600b6622b486195171f0d4a535e5a" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 35881 } } peers { permanent_uuid: "be09a9ef751642739e11c69836526ab5" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 35145 } } peers { permanent_uuid: "3c32af5cab1c4fd48fe522f4a1179c54" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 42455 } } }
I20260805 23:10:15.186611 27626 tablet_replica.cc:442] T 04d98d10ba814df7ad9a3d00860b3c2b P 3c32af5cab1c4fd48fe522f4a1179c54: This TxnStatusTablet replica's current role is: FOLLOWER
I20260805 23:10:15.191310 27617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:15.187451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58948 (local address 127.25.92.194:35145)
0805 23:10:15.187587 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:15.187593 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:15.187606 (+ 13us) server_negotiation.cc:407] Connection header received
0805 23:10:15.187649 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:15.187652 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:15.187701 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:15.187778 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 23:10:15.188069 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:15.188881 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:15.190137 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:15.190381 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:15.190493 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:15.190827 (+ 334us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 23:10:15.190949 (+ 122us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 23:10:15.191032 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:15.191156 (+ 124us) server_negotiation.cc:299] Negotiation successful
0805 23:10:15.191206 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260805 23:10:15.202879 27617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:15.199139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.194:51297 (local address 127.25.92.194:35145)
0805 23:10:15.199270 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:15.199275 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:15.199402 (+ 127us) server_negotiation.cc:407] Connection header received
0805 23:10:15.199639 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:15.199643 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:15.199696 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:15.199782 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 23:10:15.200073 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:15.200614 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:15.201615 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:15.201848 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:15.201976 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:15.202293 (+ 317us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 23:10:15.202372 (+ 79us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 23:10:15.202453 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:15.202576 (+ 123us) server_negotiation.cc:299] Negotiation successful
0805 23:10:15.202628 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":6,"server-negotiator.queue_time_us":55}
I20260805 23:10:15.202891 27670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:15.198973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.194:35145 (local address 127.25.92.194:51297)
0805 23:10:15.199298 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:15.199322 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:15.199522 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:15.199788 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:15.199791 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:15.199812 (+ 21us) client_negotiation.cc:189] Negotiated authn=TOKEN
0805 23:10:15.199961 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:15.199967 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:15.200740 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:15.200747 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:15.201422 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:15.201430 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:15.201983 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:15.201986 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:15.202095 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:15.202140 (+ 45us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0805 23:10:15.202478 (+ 338us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0805 23:10:15.202485 (+ 7us) client_negotiation.cc:769] Sending connection context
0805 23:10:15.202555 (+ 70us) client_negotiation.cc:240] Negotiation successful
0805 23:10:15.202638 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":111,"threads_started":1}
I20260805 23:10:15.210938 27261 raft_consensus.cc:1261] T f80b5a6bf5e0462ab10ef1aec4375874 P f0e600b6622b486195171f0d4a535e5a [term 1 FOLLOWER]: Refusing update from remote peer be09a9ef751642739e11c69836526ab5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:10:15.210953 27548 raft_consensus.cc:1261] T f80b5a6bf5e0462ab10ef1aec4375874 P 3c32af5cab1c4fd48fe522f4a1179c54 [term 1 FOLLOWER]: Refusing update from remote peer be09a9ef751642739e11c69836526ab5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:10:15.211367 27625 consensus_queue.cc:1035] T f80b5a6bf5e0462ab10ef1aec4375874 P be09a9ef751642739e11c69836526ab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0e600b6622b486195171f0d4a535e5a" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 35881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:10:15.211541 27650 consensus_queue.cc:1035] T f80b5a6bf5e0462ab10ef1aec4375874 P be09a9ef751642739e11c69836526ab5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c32af5cab1c4fd48fe522f4a1179c54" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 42455 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:10:15.253818 27104 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36266:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:10:15.253974 27104 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36266:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:10:15.256145 27104 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 94ae863b48d343ceb1a8d8b53a9666f7: Sending DeleteTablet for 3 replicas of tablet f80b5a6bf5e0462ab10ef1aec4375874
I20260805 23:10:15.256799 27243 tablet_service.cc:1514] Processing DeleteTablet for tablet f80b5a6bf5e0462ab10ef1aec4375874 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:10:15 UTC) from {username='kudu', principal='kudu/127.25.92.254@KRBTEST.COM'} at 127.0.0.1:55162
I20260805 23:10:15.256799 27384 tablet_service.cc:1514] Processing DeleteTablet for tablet f80b5a6bf5e0462ab10ef1aec4375874 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:10:15 UTC) from {username='kudu', principal='kudu/127.25.92.254@KRBTEST.COM'} at 127.0.0.1:58938
I20260805 23:10:15.257010 27644 tablet_replica.cc:330] T f80b5a6bf5e0462ab10ef1aec4375874 P be09a9ef751642739e11c69836526ab5: stopping tablet replica
I20260805 23:10:15.257023 27643 tablet_replica.cc:330] T f80b5a6bf5e0462ab10ef1aec4375874 P f0e600b6622b486195171f0d4a535e5a: stopping tablet replica
I20260805 23:10:15.257005 27528 tablet_service.cc:1514] Processing DeleteTablet for tablet f80b5a6bf5e0462ab10ef1aec4375874 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:10:15 UTC) from {username='kudu', principal='kudu/127.25.92.254@KRBTEST.COM'} at 127.0.0.1:56000
I20260805 23:10:15.257130 27644 raft_consensus.cc:2229] T f80b5a6bf5e0462ab10ef1aec4375874 P be09a9ef751642739e11c69836526ab5 [term 1 LEADER]: Raft consensus shutting down.
I20260805 23:10:15.257130 27643 raft_consensus.cc:2229] T f80b5a6bf5e0462ab10ef1aec4375874 P f0e600b6622b486195171f0d4a535e5a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 23:10:15.257251 27643 raft_consensus.cc:2258] T f80b5a6bf5e0462ab10ef1aec4375874 P f0e600b6622b486195171f0d4a535e5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:10:15.257433 27642 tablet_replica.cc:330] T f80b5a6bf5e0462ab10ef1aec4375874 P 3c32af5cab1c4fd48fe522f4a1179c54: stopping tablet replica
I20260805 23:10:15.257460 27644 raft_consensus.cc:2258] T f80b5a6bf5e0462ab10ef1aec4375874 P be09a9ef751642739e11c69836526ab5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:10:15.257616 27642 raft_consensus.cc:2229] T f80b5a6bf5e0462ab10ef1aec4375874 P 3c32af5cab1c4fd48fe522f4a1179c54 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 23:10:15.257761 27642 raft_consensus.cc:2258] T f80b5a6bf5e0462ab10ef1aec4375874 P 3c32af5cab1c4fd48fe522f4a1179c54 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:10:15.258324 27643 ts_tablet_manager.cc:1905] T f80b5a6bf5e0462ab10ef1aec4375874 P f0e600b6622b486195171f0d4a535e5a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:10:15.258805 27644 ts_tablet_manager.cc:1905] T f80b5a6bf5e0462ab10ef1aec4375874 P be09a9ef751642739e11c69836526ab5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:10:15.260954 27643 ts_tablet_manager.cc:1918] T f80b5a6bf5e0462ab10ef1aec4375874 P f0e600b6622b486195171f0d4a535e5a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260805 23:10:15.261009 27643 log.cc:1196] T f80b5a6bf5e0462ab10ef1aec4375874 P f0e600b6622b486195171f0d4a535e5a: Deleting WAL directory at /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/minicluster-data/ts-0/wal/wals/f80b5a6bf5e0462ab10ef1aec4375874
I20260805 23:10:15.261214 27643 ts_tablet_manager.cc:1939] T f80b5a6bf5e0462ab10ef1aec4375874 P f0e600b6622b486195171f0d4a535e5a: Deleting consensus metadata
I20260805 23:10:15.261452 27644 ts_tablet_manager.cc:1918] T f80b5a6bf5e0462ab10ef1aec4375874 P be09a9ef751642739e11c69836526ab5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260805 23:10:15.261550 27644 log.cc:1196] T f80b5a6bf5e0462ab10ef1aec4375874 P be09a9ef751642739e11c69836526ab5: Deleting WAL directory at /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/minicluster-data/ts-1/wal/wals/f80b5a6bf5e0462ab10ef1aec4375874
I20260805 23:10:15.261795 27644 ts_tablet_manager.cc:1939] T f80b5a6bf5e0462ab10ef1aec4375874 P be09a9ef751642739e11c69836526ab5: Deleting consensus metadata
I20260805 23:10:15.262240 27089 catalog_manager.cc:4841] TS f0e600b6622b486195171f0d4a535e5a (127.25.92.193:35881): tablet f80b5a6bf5e0462ab10ef1aec4375874 (table test-table [id=59396ff78e90423482e00d61ec412533]) successfully deleted
I20260805 23:10:15.262529 27088 catalog_manager.cc:4841] TS be09a9ef751642739e11c69836526ab5 (127.25.92.194:35145): tablet f80b5a6bf5e0462ab10ef1aec4375874 (table test-table [id=59396ff78e90423482e00d61ec412533]) successfully deleted
I20260805 23:10:15.263896 27642 ts_tablet_manager.cc:1905] T f80b5a6bf5e0462ab10ef1aec4375874 P 3c32af5cab1c4fd48fe522f4a1179c54: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:10:15.266695 27642 ts_tablet_manager.cc:1918] T f80b5a6bf5e0462ab10ef1aec4375874 P 3c32af5cab1c4fd48fe522f4a1179c54: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260805 23:10:15.266804 27642 log.cc:1196] T f80b5a6bf5e0462ab10ef1aec4375874 P 3c32af5cab1c4fd48fe522f4a1179c54: Deleting WAL directory at /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971406251336-25971-0/minicluster-data/ts-2/wal/wals/f80b5a6bf5e0462ab10ef1aec4375874
I20260805 23:10:15.267133 27642 ts_tablet_manager.cc:1939] T f80b5a6bf5e0462ab10ef1aec4375874 P 3c32af5cab1c4fd48fe522f4a1179c54: Deleting consensus metadata
I20260805 23:10:15.267916 27091 catalog_manager.cc:4841] TS 3c32af5cab1c4fd48fe522f4a1179c54 (127.25.92.195:42455): tablet f80b5a6bf5e0462ab10ef1aec4375874 (table test-table [id=59396ff78e90423482e00d61ec412533]) successfully deleted
Aug 05 23:10:15 dist-test-slave-ss4n krb5kdc[27058](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971414, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.25.92.193@KRBTEST.COM
I20260805 23:10:15.283073 27618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:15.261842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55178 (local address 127.25.92.193:35881)
0805 23:10:15.262271 (+ 429us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:15.262276 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:15.265850 (+ 3574us) server_negotiation.cc:407] Connection header received
0805 23:10:15.266000 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:15.266004 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:15.266064 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:15.266166 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:15.267049 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:15.267890 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:15.269165 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:15.269554 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:15.269677 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:15.278494 (+ 8817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:15.278506 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:15.278520 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:15.278552 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:15.280833 (+ 2281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:15.281436 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:15.281442 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:15.281444 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:15.281596 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:15.281927 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:15.281931 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:15.281933 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:15.282267 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:15.282511 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:15.282736 (+ 225us) server_negotiation.cc:299] Negotiation successful
0805 23:10:15.282878 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284}
W20260805 23:10:15.284042 27304 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55178
I20260805 23:10:15.299732 27154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:15.286095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36276 (local address 127.25.92.254:38411)
0805 23:10:15.286253 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:15.286258 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:15.289922 (+ 3664us) server_negotiation.cc:407] Connection header received
0805 23:10:15.290051 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:15.290055 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:15.290141 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:15.290230 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:15.291155 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:15.292113 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:15.293654 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:15.293901 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:15.294003 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:15.295885 (+ 1882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:15.295889 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:15.295891 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:15.295919 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:15.297654 (+ 1735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:15.298179 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:15.298182 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:15.298183 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:15.298265 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:15.298644 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:15.298647 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:15.298648 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:15.298834 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:15.298937 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:15.299302 (+ 365us) server_negotiation.cc:299] Negotiation successful
0805 23:10:15.299423 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
W20260805 23:10:15.300276 27104 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36276
I20260805 23:10:15.302150 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 27177
I20260805 23:10:15.310292 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 27319
I20260805 23:10:15.318102 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 27461
I20260805 23:10:15.325456 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 27071
2026-08-05T23:10:15Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2732 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 23:10:15 dist-test-slave-ss4n krb5kdc[27682](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 23:10:15 dist-test-slave-ss4n krb5kdc[27682](info): set up 2 sockets
Aug 05 23:10:15 dist-test-slave-ss4n krb5kdc[27682](info): commencing operation
krb5kdc: starting...
W20260805 23:10:16.467926 25971 mini_kdc.cc:121] Time spent starting KDC: real 1.118s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/krb5kdc/test-user.keytab.
Aug 05 23:10:16 dist-test-slave-ss4n krb5kdc[27682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971416, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T23:10:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:10:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.92.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.92.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.92.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab.
Entry for principal kudu/127.25.92.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab.
I20260805 23:10:16.606478 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.254:38325
--webserver_interface=127.25.92.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab
--principal=kudu/127.25.92.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:39723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.92.254:38325
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:10:16.722024 27695 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:10:16.722404 27695 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:10:16.722487 27695 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:10:16.726737 27695 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:10:16.726840 27695 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260805 23:10:16.726872 27695 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:10:16.726913 27695 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:10:16.726967 27695 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:10:16.727005 27695 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:10:16.732815 27695 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:39723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.92.254:38325
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab
--principal=kudu/127.25.92.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.254:38325
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.92.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27695
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:10:16.734104 27695 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:10:16.735168 27695 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:10:16.744405 27700 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:16.744418 27701 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:16.744418 27703 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:10:16.744691 27695 server_base.cc:1034] running on GCE node
I20260805 23:10:16.745384 27695 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:10:16.746570 27695 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:10:16.747758 27695 hybrid_clock.cc:648] HybridClock initialized: now 1785971416747716 us; error 64 us; skew 500 ppm
Aug 05 23:10:16 dist-test-slave-ss4n krb5kdc[27682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971416, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:10:16.750944 27695 init.cc:377] Logged in from keytab as kudu/127.25.92.254@KRBTEST.COM (short username kudu)
I20260805 23:10:16.751921 27695 webserver.cc:460] Webserver started at http://127.25.92.254:45105/ using document root <none> and password file <none>
I20260805 23:10:16.752452 27695 fs_manager.cc:359] Metadata directory not provided
I20260805 23:10:16.752512 27695 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:10:16.752691 27695 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:10:16.754886 27695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/minicluster-data/master-0/data/instance:
uuid: "ea219c710c774b9d9e270e6fad216a05"
format_stamp: "Formatted at 2026-08-05 23:10:16 on dist-test-slave-ss4n"
server_key: "e9d986b67b1cfcc48b3ca0e4f8b73919"
server_key_iv: "fc4e9532944db265990b5ef868e7be86"
server_key_version: "encryptionkey@0"
I20260805 23:10:16.755483 27695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/minicluster-data/master-0/wal/instance:
uuid: "ea219c710c774b9d9e270e6fad216a05"
format_stamp: "Formatted at 2026-08-05 23:10:16 on dist-test-slave-ss4n"
server_key: "e9d986b67b1cfcc48b3ca0e4f8b73919"
server_key_iv: "fc4e9532944db265990b5ef868e7be86"
server_key_version: "encryptionkey@0"
I20260805 23:10:16.759531 27695 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260805 23:10:16.762236 27710 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:16.763566 27695 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:10:16.763741 27695 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/minicluster-data/master-0/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/minicluster-data/master-0/wal
uuid: "ea219c710c774b9d9e270e6fad216a05"
format_stamp: "Formatted at 2026-08-05 23:10:16 on dist-test-slave-ss4n"
server_key: "e9d986b67b1cfcc48b3ca0e4f8b73919"
server_key_iv: "fc4e9532944db265990b5ef868e7be86"
server_key_version: "encryptionkey@0"
I20260805 23:10:16.763868 27695 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:10:16.779964 27695 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:10:16.783339 27695 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:10:16.783591 27695 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:10:16.792250 27695 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.254:38325
I20260805 23:10:16.792313 27773 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.254:38325 every 8 connection(s)
I20260805 23:10:16.793244 27695 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
I20260805 23:10:16.793671 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 27695
I20260805 23:10:16.793805 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/minicluster-data/master-0/wal/instance
I20260805 23:10:16.794165 25971 external_mini_cluster.cc:1428] Setting key c3f3ac9c5136d6eea1168aced29d1333
I20260805 23:10:16.796849 27774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 05 23:10:16 dist-test-slave-ss4n krb5kdc[27682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971416, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM
I20260805 23:10:16.803435 27774 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea219c710c774b9d9e270e6fad216a05: Bootstrap starting.
I20260805 23:10:16.806294 27774 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ea219c710c774b9d9e270e6fad216a05: Neither blocks nor log segments found. Creating new log.
I20260805 23:10:16.807670 27774 log.cc:824] T 00000000000000000000000000000000 P ea219c710c774b9d9e270e6fad216a05: Log is configured to *not* fsync() on all Append() calls
I20260805 23:10:16.809041 27778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:16.795707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52116 (local address 127.25.92.254:38325)
0805 23:10:16.796203 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:16.796218 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:16.796254 (+ 36us) server_negotiation.cc:407] Connection header received
0805 23:10:16.796993 (+ 739us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:16.797020 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:16.797376 (+ 356us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:16.797788 (+ 412us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:16.798603 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:16.799617 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:16.800486 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:16.800964 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:16.801148 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:16.803940 (+ 2792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:16.803944 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:16.803958 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:16.803990 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:16.805906 (+ 1916us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:16.806376 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:16.806382 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:16.806386 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:16.806553 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:16.806841 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:16.806846 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:16.806848 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:16.807282 (+ 434us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:16.807483 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:16.807793 (+ 310us) server_negotiation.cc:299] Negotiation successful
0805 23:10:16.808113 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":153,"threads_started":1}
I20260805 23:10:16.810233 27774 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea219c710c774b9d9e270e6fad216a05: No bootstrap required, opened a new log
I20260805 23:10:16.813819 27774 raft_consensus.cc:348] T 00000000000000000000000000000000 P ea219c710c774b9d9e270e6fad216a05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea219c710c774b9d9e270e6fad216a05" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 38325 } }
I20260805 23:10:16.814082 27774 raft_consensus.cc:374] T 00000000000000000000000000000000 P ea219c710c774b9d9e270e6fad216a05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:10:16.814160 27774 raft_consensus.cc:729] T 00000000000000000000000000000000 P ea219c710c774b9d9e270e6fad216a05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea219c710c774b9d9e270e6fad216a05, State: Initialized, Role: FOLLOWER
I20260805 23:10:16.814671 27774 consensus_queue.cc:260] T 00000000000000000000000000000000 P ea219c710c774b9d9e270e6fad216a05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea219c710c774b9d9e270e6fad216a05" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 38325 } }
I20260805 23:10:16.814833 27774 raft_consensus.cc:388] T 00000000000000000000000000000000 P ea219c710c774b9d9e270e6fad216a05 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:10:16.814906 27774 raft_consensus.cc:482] T 00000000000000000000000000000000 P ea219c710c774b9d9e270e6fad216a05 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:10:16.815019 27774 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ea219c710c774b9d9e270e6fad216a05 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:10:16.816107 27774 raft_consensus.cc:504] T 00000000000000000000000000000000 P ea219c710c774b9d9e270e6fad216a05 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea219c710c774b9d9e270e6fad216a05" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 38325 } }
I20260805 23:10:16.816484 27774 leader_election.cc:302] T 00000000000000000000000000000000 P ea219c710c774b9d9e270e6fad216a05 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ea219c710c774b9d9e270e6fad216a05; no voters:
I20260805 23:10:16.816852 27774 leader_election.cc:288] T 00000000000000000000000000000000 P ea219c710c774b9d9e270e6fad216a05 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 23:10:16.817023 27780 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ea219c710c774b9d9e270e6fad216a05 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:10:16.817317 27780 raft_consensus.cc:686] T 00000000000000000000000000000000 P ea219c710c774b9d9e270e6fad216a05 [term 1 LEADER]: Becoming Leader. State: Replica: ea219c710c774b9d9e270e6fad216a05, State: Running, Role: LEADER
I20260805 23:10:16.817667 27780 consensus_queue.cc:237] T 00000000000000000000000000000000 P ea219c710c774b9d9e270e6fad216a05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea219c710c774b9d9e270e6fad216a05" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 38325 } }
I20260805 23:10:16.818328 27774 sys_catalog.cc:564] T 00000000000000000000000000000000 P ea219c710c774b9d9e270e6fad216a05 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:10:16.819587 27781 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea219c710c774b9d9e270e6fad216a05 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ea219c710c774b9d9e270e6fad216a05" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea219c710c774b9d9e270e6fad216a05" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 38325 } } }
I20260805 23:10:16.819715 27781 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea219c710c774b9d9e270e6fad216a05 [sys.catalog]: This master's current role is: LEADER
I20260805 23:10:16.819957 27782 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea219c710c774b9d9e270e6fad216a05 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ea219c710c774b9d9e270e6fad216a05. Latest consensus state: current_term: 1 leader_uuid: "ea219c710c774b9d9e270e6fad216a05" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea219c710c774b9d9e270e6fad216a05" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 38325 } } }
I20260805 23:10:16.820044 27782 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea219c710c774b9d9e270e6fad216a05 [sys.catalog]: This master's current role is: LEADER
I20260805 23:10:16.820655 27789 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:10:16.823910 27789 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:10:16.829648 27789 catalog_manager.cc:1347] Generated new cluster ID: 008977c7b70947d9a3a137d7da5d309a
I20260805 23:10:16.829739 27789 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:10:16.870971 27789 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:10:16.872021 27789 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:10:16.879969 27789 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ea219c710c774b9d9e270e6fad216a05: Generated new TSK 0
I20260805 23:10:16.880779 27789 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.92.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.92.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.92.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.193.keytab.
Entry for principal kudu/127.25.92.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.193.keytab.
I20260805 23:10:16.922930 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.92.193:0
--local_ip_for_outbound_sockets=127.25.92.193
--webserver_interface=127.25.92.193
--webserver_port=0
--tserver_master_addrs=127.25.92.254:38325
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.193.keytab
--principal=kudu/127.25.92.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:39723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:10:17.053314 27802 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:10:17.053735 27802 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:10:17.053797 27802 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:10:17.058152 27802 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:10:17.058218 27802 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:10:17.058262 27802 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260805 23:10:17.058277 27802 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260805 23:10:17.058311 27802 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.92.193
I20260805 23:10:17.063983 27802 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:39723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.193.keytab
--principal=kudu/127.25.92.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.92.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.92.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.25.92.254:38325
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27802
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.92.193
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:10:17.065085 27802 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:10:17.066089 27802 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:10:17.076715 27808 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:10:17.076763 27802 server_base.cc:1034] running on GCE node
W20260805 23:10:17.076856 27807 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:17.076792 27810 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:10:17.077603 27802 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:10:17.078401 27802 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:10:17.079645 27802 hybrid_clock.cc:648] HybridClock initialized: now 1785971417079628 us; error 37 us; skew 500 ppm
Aug 05 23:10:17 dist-test-slave-ss4n krb5kdc[27682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971417, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:10:17.082935 27802 init.cc:377] Logged in from keytab as kudu/127.25.92.193@KRBTEST.COM (short username kudu)
I20260805 23:10:17.083930 27802 webserver.cc:460] Webserver started at http://127.25.92.193:35805/ using document root <none> and password file <none>
I20260805 23:10:17.084547 27802 fs_manager.cc:359] Metadata directory not provided
I20260805 23:10:17.084645 27802 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:10:17.084882 27802 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:10:17.087046 27802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/minicluster-data/ts-0/data/instance:
uuid: "a2b8471244eb4bb4825f255d21e3cdba"
format_stamp: "Formatted at 2026-08-05 23:10:17 on dist-test-slave-ss4n"
server_key: "8222523368820d9448ec83304472a7df"
server_key_iv: "9ba27834494b8664b97c837525dc5336"
server_key_version: "encryptionkey@0"
I20260805 23:10:17.087628 27802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/minicluster-data/ts-0/wal/instance:
uuid: "a2b8471244eb4bb4825f255d21e3cdba"
format_stamp: "Formatted at 2026-08-05 23:10:17 on dist-test-slave-ss4n"
server_key: "8222523368820d9448ec83304472a7df"
server_key_iv: "9ba27834494b8664b97c837525dc5336"
server_key_version: "encryptionkey@0"
I20260805 23:10:17.091679 27802 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260805 23:10:17.094580 27817 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:17.096269 27802 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:10:17.096422 27802 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/minicluster-data/ts-0/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/minicluster-data/ts-0/wal
uuid: "a2b8471244eb4bb4825f255d21e3cdba"
format_stamp: "Formatted at 2026-08-05 23:10:17 on dist-test-slave-ss4n"
server_key: "8222523368820d9448ec83304472a7df"
server_key_iv: "9ba27834494b8664b97c837525dc5336"
server_key_version: "encryptionkey@0"
I20260805 23:10:17.096627 27802 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:10:17.118394 27802 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:10:17.122241 27802 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:10:17.122488 27802 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:10:17.124197 27802 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:10:17.124251 27802 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:17.124295 27802 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:10:17.124361 27802 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 05 23:10:17 dist-test-slave-ss4n krb5kdc[27682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971417, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.193@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM
I20260805 23:10:17.136950 27802 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.193:34855
I20260805 23:10:17.137321 27933 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.193:34855 every 8 connection(s)
I20260805 23:10:17.137993 27802 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/minicluster-data/ts-0/data/info.pb
I20260805 23:10:17.140736 27778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:17.125232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.193:58363 (local address 127.25.92.254:38325)
0805 23:10:17.125420 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:17.125428 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:17.126470 (+ 1042us) server_negotiation.cc:407] Connection header received
0805 23:10:17.127884 (+ 1414us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:17.127888 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:17.127988 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:17.128131 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:17.130013 (+ 1882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:17.130979 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:17.132377 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:17.132827 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:17.132986 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:17.136482 (+ 3496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:17.136492 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:17.136494 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:17.136528 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:17.138403 (+ 1875us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:17.139087 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:17.139091 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:17.139093 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:17.139181 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:17.139545 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:17.139548 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:17.139550 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:17.139751 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:17.139869 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:17.140384 (+ 515us) server_negotiation.cc:299] Negotiation successful
0805 23:10:17.140536 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260805 23:10:17.141446 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 27802
I20260805 23:10:17.141618 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/minicluster-data/ts-0/wal/instance
I20260805 23:10:17.141620 27826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:17.125694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:38325 (local address 127.25.92.193:58363)
0805 23:10:17.126263 (+ 569us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:17.126329 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:17.127599 (+ 1270us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:17.128359 (+ 760us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:17.128370 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:17.128945 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:17.129808 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:17.129835 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:17.131182 (+ 1347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:17.131186 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:17.132242 (+ 1056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:17.132252 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:17.133076 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:17.133085 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:17.133371 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:17.134113 (+ 742us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:10:17.134138 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:17.136259 (+ 2121us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:10:17.138566 (+ 2307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:17.138572 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:17.138587 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:17.138965 (+ 378us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:17.139280 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:17.139284 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:17.139287 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:17.139433 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:17.139865 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:10:17.139871 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:10:17.140132 (+ 261us) client_negotiation.cc:769] Sending connection context
0805 23:10:17.140315 (+ 183us) client_negotiation.cc:240] Negotiation successful
0805 23:10:17.140647 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":120,"threads_started":1}
I20260805 23:10:17.141981 25971 external_mini_cluster.cc:1428] Setting key a808781942a827be62c6a91a6e588df5
I20260805 23:10:17.143167 27934 heartbeater.cc:347] Connected to a master server at 127.25.92.254:38325
I20260805 23:10:17.143306 27934 heartbeater.cc:464] Registering TS with master...
I20260805 23:10:17.143993 27934 heartbeater.cc:511] Master 127.25.92.254:38325 requested a full tablet report, sending...
I20260805 23:10:17.145758 27727 ts_manager.cc:194] Registered new tserver with Master: a2b8471244eb4bb4825f255d21e3cdba (127.25.92.193:34855)
I20260805 23:10:17.147130 27727 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.92.193@KRBTEST.COM'} at 127.25.92.193:58363
WARNING: no policy specified for kudu/127.25.92.194@KRBTEST.COM; defaulting to no policy
I20260805 23:10:17.156809 27941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:17.145459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:38325 (local address 127.25.92.193:53517)
0805 23:10:17.147976 (+ 2517us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:17.147997 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:17.148149 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:17.148506 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:17.148512 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:17.148867 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:17.149047 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:17.149059 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:17.150433 (+ 1374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:17.150438 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:17.151455 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:17.151466 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:17.152139 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:17.152145 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:17.152258 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:17.152809 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:10:17.152829 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:17.153307 (+ 478us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:10:17.155205 (+ 1898us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:17.155207 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:17.155209 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:17.155404 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:17.155654 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:17.155656 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:17.155657 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:17.155730 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:17.156068 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:10:17.156070 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:10:17.156201 (+ 131us) client_negotiation.cc:769] Sending connection context
0805 23:10:17.156398 (+ 197us) client_negotiation.cc:240] Negotiation successful
0805 23:10:17.156560 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2391,"thread_start_us":97,"threads_started":1}
I20260805 23:10:17.156797 27778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:17.145601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.193:53517 (local address 127.25.92.254:38325)
0805 23:10:17.145725 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:17.145729 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:17.148098 (+ 2369us) server_negotiation.cc:407] Connection header received
0805 23:10:17.148267 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:17.148270 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:17.148338 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:17.148434 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:17.149232 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:17.150201 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:17.151731 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:17.151996 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:17.152092 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:17.153461 (+ 1369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:17.153464 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:17.153466 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:17.153555 (+ 89us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:17.155091 (+ 1536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:17.155492 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:17.155494 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:17.155495 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:17.155571 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:17.155807 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:17.155809 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:17.155810 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:17.155980 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:17.156062 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:17.156464 (+ 402us) server_negotiation.cc:299] Negotiation successful
0805 23:10:17.156644 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.92.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.92.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.194.keytab.
Entry for principal kudu/127.25.92.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.194.keytab.
I20260805 23:10:17.173285 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.92.194:0
--local_ip_for_outbound_sockets=127.25.92.194
--webserver_interface=127.25.92.194
--webserver_port=0
--tserver_master_addrs=127.25.92.254:38325
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.194.keytab
--principal=kudu/127.25.92.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:39723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:10:17.291499 27944 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:10:17.291807 27944 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:10:17.291857 27944 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:10:17.295974 27944 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:10:17.296053 27944 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:10:17.296094 27944 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260805 23:10:17.296109 27944 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260805 23:10:17.296185 27944 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.92.194
I20260805 23:10:17.301828 27944 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:39723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.194.keytab
--principal=kudu/127.25.92.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.92.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.92.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.25.92.254:38325
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27944
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.92.194
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:10:17.302937 27944 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:10:17.303969 27944 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:10:17.314321 27953 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:17.314349 27950 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:17.314395 27949 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:10:17.314482 27944 server_base.cc:1034] running on GCE node
I20260805 23:10:17.315006 27944 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:10:17.315763 27944 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:10:17.316960 27944 hybrid_clock.cc:648] HybridClock initialized: now 1785971417316943 us; error 30 us; skew 500 ppm
Aug 05 23:10:17 dist-test-slave-ss4n krb5kdc[27682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971417, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:10:17.320091 27944 init.cc:377] Logged in from keytab as kudu/127.25.92.194@KRBTEST.COM (short username kudu)
I20260805 23:10:17.321170 27944 webserver.cc:460] Webserver started at http://127.25.92.194:36447/ using document root <none> and password file <none>
I20260805 23:10:17.321823 27944 fs_manager.cc:359] Metadata directory not provided
I20260805 23:10:17.321890 27944 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:10:17.322062 27944 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:10:17.324169 27944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/minicluster-data/ts-1/data/instance:
uuid: "3fa2896d2f224c69b1bb196eb4619a17"
format_stamp: "Formatted at 2026-08-05 23:10:17 on dist-test-slave-ss4n"
server_key: "9e34047a43285f533bc7585a8d4bb719"
server_key_iv: "d24c38d488edff88cb049fa0088a4ad2"
server_key_version: "encryptionkey@0"
I20260805 23:10:17.324723 27944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/minicluster-data/ts-1/wal/instance:
uuid: "3fa2896d2f224c69b1bb196eb4619a17"
format_stamp: "Formatted at 2026-08-05 23:10:17 on dist-test-slave-ss4n"
server_key: "9e34047a43285f533bc7585a8d4bb719"
server_key_iv: "d24c38d488edff88cb049fa0088a4ad2"
server_key_version: "encryptionkey@0"
I20260805 23:10:17.328727 27944 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260805 23:10:17.331666 27960 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:17.333137 27944 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260805 23:10:17.333261 27944 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/minicluster-data/ts-1/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/minicluster-data/ts-1/wal
uuid: "3fa2896d2f224c69b1bb196eb4619a17"
format_stamp: "Formatted at 2026-08-05 23:10:17 on dist-test-slave-ss4n"
server_key: "9e34047a43285f533bc7585a8d4bb719"
server_key_iv: "d24c38d488edff88cb049fa0088a4ad2"
server_key_version: "encryptionkey@0"
I20260805 23:10:17.333436 27944 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:10:17.350898 27944 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:10:17.354691 27944 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:10:17.354876 27944 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:10:17.356426 27944 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:10:17.356487 27944 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:17.356535 27944 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:10:17.356587 27944 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 05 23:10:17 dist-test-slave-ss4n krb5kdc[27682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971417, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.194@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM
I20260805 23:10:17.369174 27944 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.194:42229
I20260805 23:10:17.369302 28076 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.194:42229 every 8 connection(s)
I20260805 23:10:17.370395 27944 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/minicluster-data/ts-1/data/info.pb
I20260805 23:10:17.373764 27778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:17.357666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.194:37619 (local address 127.25.92.254:38325)
0805 23:10:17.357823 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:17.357827 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:17.359018 (+ 1191us) server_negotiation.cc:407] Connection header received
0805 23:10:17.360621 (+ 1603us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:17.360625 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:17.360698 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:17.360824 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:17.362291 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:17.362904 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:17.364529 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:17.364801 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:17.364950 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:17.369409 (+ 4459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:17.369416 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:17.369419 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:17.369470 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:17.371424 (+ 1954us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:17.372051 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:17.372055 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:17.372056 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:17.372145 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:17.372523 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:17.372527 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:17.372528 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:17.372733 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:17.372843 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:17.373385 (+ 542us) server_negotiation.cc:299] Negotiation successful
0805 23:10:17.373588 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260805 23:10:17.374543 27969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:17.358193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:38325 (local address 127.25.92.194:37619)
0805 23:10:17.358796 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:17.358856 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:17.360346 (+ 1490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:17.361028 (+ 682us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:17.361036 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:17.361470 (+ 434us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:17.362088 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:17.362103 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:17.363063 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:17.363068 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:17.364364 (+ 1296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:17.364375 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:17.365015 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:17.365020 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:17.365229 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:17.366897 (+ 1668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:10:17.366936 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:17.369117 (+ 2181us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:10:17.371626 (+ 2509us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:17.371632 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:17.371647 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:17.371920 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:17.372262 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:17.372265 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:17.372267 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:17.372411 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:17.372844 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:10:17.372851 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:10:17.373110 (+ 259us) client_negotiation.cc:769] Sending connection context
0805 23:10:17.373319 (+ 209us) client_negotiation.cc:240] Negotiation successful
0805 23:10:17.373717 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":140,"threads_started":1}
I20260805 23:10:17.375927 28077 heartbeater.cc:347] Connected to a master server at 127.25.92.254:38325
I20260805 23:10:17.376082 28077 heartbeater.cc:464] Registering TS with master...
I20260805 23:10:17.376734 28077 heartbeater.cc:511] Master 127.25.92.254:38325 requested a full tablet report, sending...
I20260805 23:10:17.378008 27727 ts_manager.cc:194] Registered new tserver with Master: 3fa2896d2f224c69b1bb196eb4619a17 (127.25.92.194:42229)
I20260805 23:10:17.378801 27727 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.92.194@KRBTEST.COM'} at 127.25.92.194:37619
I20260805 23:10:17.380493 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 27944
I20260805 23:10:17.380683 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/minicluster-data/ts-1/wal/instance
I20260805 23:10:17.381026 25971 external_mini_cluster.cc:1428] Setting key b41e2e506902757911ed7270a7619d33
I20260805 23:10:17.388020 28083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:17.378456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:38325 (local address 127.25.92.194:39911)
0805 23:10:17.378751 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:17.378772 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:17.378893 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:17.379272 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:17.379300 (+ 28us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:17.379541 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:17.379675 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:17.379682 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:17.380852 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:17.380856 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:17.381664 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:17.381674 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:17.382274 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:17.382277 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:17.382409 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:17.383183 (+ 774us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:10:17.383200 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:17.383584 (+ 384us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:10:17.385973 (+ 2389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:17.385976 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:17.385978 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:17.386219 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:17.386611 (+ 392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:17.386614 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:17.386616 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:17.386709 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:17.387368 (+ 659us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:10:17.387374 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:10:17.387576 (+ 202us) client_negotiation.cc:769] Sending connection context
0805 23:10:17.387691 (+ 115us) client_negotiation.cc:240] Negotiation successful
0805 23:10:17.387828 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":108,"threads_started":1}
I20260805 23:10:17.388125 27778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:17.378581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.194:39911 (local address 127.25.92.254:38325)
0805 23:10:17.378724 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:17.378729 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:17.378842 (+ 113us) server_negotiation.cc:407] Connection header received
0805 23:10:17.379016 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:17.379020 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:17.379074 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:17.379163 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:17.379784 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:17.380649 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:17.381791 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:17.382122 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:17.382241 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:17.383746 (+ 1505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:17.383753 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:17.383759 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:17.383792 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:17.385815 (+ 2023us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:17.386356 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:17.386363 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:17.386368 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:17.386471 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:17.386821 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:17.386827 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:17.386832 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:17.387196 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:17.387313 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:17.387782 (+ 469us) server_negotiation.cc:299] Negotiation successful
0805 23:10:17.387924 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
WARNING: no policy specified for kudu/127.25.92.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.92.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.92.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.195.keytab.
Entry for principal kudu/127.25.92.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.195.keytab.
I20260805 23:10:17.413214 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.92.195:0
--local_ip_for_outbound_sockets=127.25.92.195
--webserver_interface=127.25.92.195
--webserver_port=0
--tserver_master_addrs=127.25.92.254:38325
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.195.keytab
--principal=kudu/127.25.92.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:39723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:10:17.532850 28086 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:10:17.533164 28086 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:10:17.533219 28086 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:10:17.538079 28086 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:10:17.538156 28086 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:10:17.538226 28086 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260805 23:10:17.538264 28086 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260805 23:10:17.538319 28086 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.92.195
I20260805 23:10:17.544886 28086 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:39723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.195.keytab
--principal=kudu/127.25.92.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.92.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.92.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.25.92.254:38325
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28086
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.92.195
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:10:17.546151 28086 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:10:17.547245 28086 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:10:17.558586 28094 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:17.558741 28091 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:10:17.558794 28086 server_base.cc:1034] running on GCE node
W20260805 23:10:17.558653 28092 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:10:17.559310 28086 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:10:17.559979 28086 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:10:17.561152 28086 hybrid_clock.cc:648] HybridClock initialized: now 1785971417561133 us; error 32 us; skew 500 ppm
Aug 05 23:10:17 dist-test-slave-ss4n krb5kdc[27682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971417, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:10:17.564415 28086 init.cc:377] Logged in from keytab as kudu/127.25.92.195@KRBTEST.COM (short username kudu)
I20260805 23:10:17.565658 28086 webserver.cc:460] Webserver started at http://127.25.92.195:44039/ using document root <none> and password file <none>
I20260805 23:10:17.566298 28086 fs_manager.cc:359] Metadata directory not provided
I20260805 23:10:17.566365 28086 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:10:17.566618 28086 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:10:17.569031 28086 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/minicluster-data/ts-2/data/instance:
uuid: "ee3ec60b51b44ca995c23c3908f540b5"
format_stamp: "Formatted at 2026-08-05 23:10:17 on dist-test-slave-ss4n"
server_key: "c291d81c071ab6cfc87e788409a0a99e"
server_key_iv: "5d90400edde923cfd0a7bfeddffbf402"
server_key_version: "encryptionkey@0"
I20260805 23:10:17.569677 28086 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/minicluster-data/ts-2/wal/instance:
uuid: "ee3ec60b51b44ca995c23c3908f540b5"
format_stamp: "Formatted at 2026-08-05 23:10:17 on dist-test-slave-ss4n"
server_key: "c291d81c071ab6cfc87e788409a0a99e"
server_key_iv: "5d90400edde923cfd0a7bfeddffbf402"
server_key_version: "encryptionkey@0"
I20260805 23:10:17.573993 28086 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260805 23:10:17.576674 28101 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:17.578064 28086 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:10:17.578174 28086 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/minicluster-data/ts-2/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/minicluster-data/ts-2/wal
uuid: "ee3ec60b51b44ca995c23c3908f540b5"
format_stamp: "Formatted at 2026-08-05 23:10:17 on dist-test-slave-ss4n"
server_key: "c291d81c071ab6cfc87e788409a0a99e"
server_key_iv: "5d90400edde923cfd0a7bfeddffbf402"
server_key_version: "encryptionkey@0"
I20260805 23:10:17.578271 28086 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:10:17.592432 28086 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:10:17.596211 28086 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:10:17.596463 28086 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:10:17.598119 28086 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:10:17.598172 28086 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:17.598212 28086 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:10:17.598263 28086 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 05 23:10:17 dist-test-slave-ss4n krb5kdc[27682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971417, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.195@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM
I20260805 23:10:17.609206 28086 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.195:38893
I20260805 23:10:17.609428 28217 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.195:38893 every 8 connection(s)
I20260805 23:10:17.610402 28086 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/minicluster-data/ts-2/data/info.pb
I20260805 23:10:17.611322 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 28086
I20260805 23:10:17.611438 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/minicluster-data/ts-2/wal/instance
I20260805 23:10:17.611768 25971 external_mini_cluster.cc:1428] Setting key e8bbf2362d309ce5e25452ae238a83b4
I20260805 23:10:17.612835 27778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:17.599002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.195:35379 (local address 127.25.92.254:38325)
0805 23:10:17.599168 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:17.599172 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:17.600152 (+ 980us) server_negotiation.cc:407] Connection header received
0805 23:10:17.601390 (+ 1238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:17.601394 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:17.601504 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:17.601591 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:17.603091 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:17.603631 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:17.604960 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:17.605193 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:17.605310 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:17.608547 (+ 3237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:17.608551 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:17.608556 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:17.608589 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:17.610440 (+ 1851us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:17.611086 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:17.611091 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:17.611094 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:17.611201 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:17.611648 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:17.611652 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:17.611654 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:17.611910 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:17.612022 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:17.612533 (+ 511us) server_negotiation.cc:299] Negotiation successful
0805 23:10:17.612668 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260805 23:10:17.613718 28110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:17.599460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:38325 (local address 127.25.92.195:35379)
0805 23:10:17.599984 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:17.600026 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:17.601166 (+ 1140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:17.601935 (+ 769us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:17.601945 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:17.602382 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:17.602932 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:17.602947 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:17.603759 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:17.603763 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:17.604820 (+ 1057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:17.604830 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:17.605363 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:17.605367 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:17.605611 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:17.606318 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:10:17.606343 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:17.608331 (+ 1988us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:10:17.610594 (+ 2263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:17.610602 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:17.610616 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:17.610979 (+ 363us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:17.611320 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:17.611324 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:17.611326 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:17.611527 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:17.612036 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:10:17.612045 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:10:17.612285 (+ 240us) client_negotiation.cc:769] Sending connection context
0805 23:10:17.612480 (+ 195us) client_negotiation.cc:240] Negotiation successful
0805 23:10:17.612811 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":144,"threads_started":1}
I20260805 23:10:17.614907 28218 heartbeater.cc:347] Connected to a master server at 127.25.92.254:38325
I20260805 23:10:17.615075 28218 heartbeater.cc:464] Registering TS with master...
I20260805 23:10:17.615564 28218 heartbeater.cc:511] Master 127.25.92.254:38325 requested a full tablet report, sending...
I20260805 23:10:17.616591 27728 ts_manager.cc:194] Registered new tserver with Master: ee3ec60b51b44ca995c23c3908f540b5 (127.25.92.195:38893)
I20260805 23:10:17.617308 27728 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.92.195@KRBTEST.COM'} at 127.25.92.195:35379
I20260805 23:10:17.624761 27778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:17.617154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.195:36691 (local address 127.25.92.254:38325)
0805 23:10:17.617263 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:17.617267 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:17.617521 (+ 254us) server_negotiation.cc:407] Connection header received
0805 23:10:17.617660 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:17.617663 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:17.617707 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:17.617794 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:17.618227 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:17.618982 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:17.619955 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:17.620209 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:17.620310 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:17.621598 (+ 1288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:17.621602 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:17.621605 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:17.621630 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:17.623182 (+ 1552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:17.623608 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:17.623610 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:17.623612 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:17.623683 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:17.623915 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:17.623917 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:17.623918 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:17.624097 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:17.624171 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:17.624490 (+ 319us) server_negotiation.cc:299] Negotiation successful
0805 23:10:17.624589 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260805 23:10:17.624807 28224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:17.617104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:38325 (local address 127.25.92.195:36691)
0805 23:10:17.617394 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:17.617412 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:17.617552 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:17.617820 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:17.617823 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:17.617993 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:17.618111 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:17.618117 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:17.619111 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:17.619114 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:17.619817 (+ 703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:17.619827 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:17.620320 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:17.620323 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:17.620422 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:17.621042 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:10:17.621056 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:17.621373 (+ 317us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:10:17.623295 (+ 1922us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:17.623298 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:17.623299 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:17.623493 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:17.623763 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:17.623765 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:17.623766 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:17.623840 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:17.624194 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:10:17.624197 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:10:17.624303 (+ 106us) client_negotiation.cc:769] Sending connection context
0805 23:10:17.624396 (+ 93us) client_negotiation.cc:240] Negotiation successful
0805 23:10:17.624614 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":103,"threads_started":1}
I20260805 23:10:17.626039 25971 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 05 23:10:17 dist-test-slave-ss4n krb5kdc[27682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971417, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 05 23:10:17 dist-test-slave-ss4n krb5kdc[27682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971417, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM
I20260805 23:10:17.653959 27778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:17.645072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52132 (local address 127.25.92.254:38325)
0805 23:10:17.645246 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:17.645249 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:17.645262 (+ 13us) server_negotiation.cc:407] Connection header received
0805 23:10:17.645304 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:17.645306 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:17.645345 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:17.645442 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:17.646242 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:17.646769 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:17.647671 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:17.647889 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:17.648014 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:17.650637 (+ 2623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:17.650640 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:17.650642 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:17.650670 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:17.651996 (+ 1326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:17.652407 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:17.652409 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:17.652411 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:17.652483 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:17.652838 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:17.652842 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:17.652843 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:17.653219 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:17.653374 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:17.653699 (+ 325us) server_negotiation.cc:299] Negotiation successful
0805 23:10:17.653814 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260805 23:10:17.657348 27727 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52132:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260805 23:10:17.660905 27727 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260805 23:10:17.676803 28239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:17.671139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.194:42229 (local address 127.0.0.1:33210)
0805 23:10:17.672033 (+ 894us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:17.672044 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:17.672168 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:17.672606 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:17.672608 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:17.672629 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:10:17.672719 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:17.672725 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:17.674040 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:17.674043 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:17.675016 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:17.675025 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:17.676204 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:17.676208 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:17.676321 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:17.676368 (+ 47us) client_negotiation.cc:769] Sending connection context
0805 23:10:17.676478 (+ 110us) client_negotiation.cc:240] Negotiation successful
0805 23:10:17.676635 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":840,"mutex_wait_us":1,"thread_start_us":91,"threads_started":1}
I20260805 23:10:17.676980 28240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:17.671254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49550 (local address 127.25.92.193:34855)
0805 23:10:17.672407 (+ 1153us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:17.672414 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:17.672435 (+ 21us) server_negotiation.cc:407] Connection header received
0805 23:10:17.672506 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:17.672513 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:17.672679 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:17.672807 (+ 128us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:10:17.673210 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:17.674298 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:17.675667 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:17.676340 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:17.676511 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:17.676692 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:17.676745 (+ 53us) server_negotiation.cc:299] Negotiation successful
0805 23:10:17.676844 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1054,"thread_start_us":87,"threads_started":1}
I20260805 23:10:17.677306 28241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:17.671583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33210 (local address 127.25.92.194:42229)
0805 23:10:17.671909 (+ 326us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:17.671920 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:17.672145 (+ 225us) server_negotiation.cc:407] Connection header received
0805 23:10:17.672280 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:17.672286 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:17.672467 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:17.672625 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:10:17.672815 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:17.673903 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:17.675187 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:17.676070 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:17.676185 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:17.676361 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:17.677050 (+ 689us) server_negotiation.cc:299] Negotiation successful
0805 23:10:17.677169 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":229,"thread_start_us":139,"threads_started":1}
I20260805 23:10:17.677462 28236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:17.670400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.195:38893 (local address 127.0.0.1:46668)
0805 23:10:17.670942 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:17.670997 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:17.671138 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:17.672232 (+ 1094us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:17.672238 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:17.672261 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:10:17.672409 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:17.672416 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:17.673777 (+ 1361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:17.673780 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:17.675134 (+ 1354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:17.675144 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:17.676430 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:17.676434 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:17.676636 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:17.676654 (+ 18us) client_negotiation.cc:769] Sending connection context
0805 23:10:17.676715 (+ 61us) client_negotiation.cc:240] Negotiation successful
0805 23:10:17.677316 (+ 601us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"mutex_wait_us":592,"thread_start_us":91,"threads_started":1}
I20260805 23:10:17.677858 28237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:17.670622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46668 (local address 127.25.92.195:38893)
0805 23:10:17.671828 (+ 1206us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:17.671837 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:17.671856 (+ 19us) server_negotiation.cc:407] Connection header received
0805 23:10:17.671917 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:17.671920 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:17.672082 (+ 162us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:17.672243 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:10:17.672524 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:17.673643 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:17.675242 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:17.676070 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:17.676185 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:17.676361 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:17.676834 (+ 473us) server_negotiation.cc:299] Negotiation successful
0805 23:10:17.677724 (+ 890us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1117,"thread_start_us":120,"threads_started":1}
I20260805 23:10:17.678041 28238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:17.670780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.193:34855 (local address 127.0.0.1:49550)
0805 23:10:17.671506 (+ 726us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:17.671519 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:17.671619 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:17.672952 (+ 1333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:17.672955 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:17.672972 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:10:17.673062 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:17.673068 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:17.674434 (+ 1366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:17.674437 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:17.675555 (+ 1118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:17.675565 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:17.676439 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:17.676442 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:17.676578 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:17.676636 (+ 58us) client_negotiation.cc:769] Sending connection context
0805 23:10:17.676705 (+ 69us) client_negotiation.cc:240] Negotiation successful
0805 23:10:17.676794 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":659,"mutex_wait_us":25,"thread_start_us":109,"threads_started":1}
I20260805 23:10:17.680236 28152 tablet_service.cc:1467] Processing CreateTablet for tablet 91f3ece73a5a47839322fdabdc40cc52 (DEFAULT_TABLE table=test-table [id=69d9b3bdd65648418911b2ec3e39bd84]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:10:17.680363 28010 tablet_service.cc:1467] Processing CreateTablet for tablet 91f3ece73a5a47839322fdabdc40cc52 (DEFAULT_TABLE table=test-table [id=69d9b3bdd65648418911b2ec3e39bd84]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:10:17.680383 27868 tablet_service.cc:1467] Processing CreateTablet for tablet 91f3ece73a5a47839322fdabdc40cc52 (DEFAULT_TABLE table=test-table [id=69d9b3bdd65648418911b2ec3e39bd84]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:10:17.681334 28152 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 91f3ece73a5a47839322fdabdc40cc52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:10:17.681365 28010 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 91f3ece73a5a47839322fdabdc40cc52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:10:17.681365 27868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 91f3ece73a5a47839322fdabdc40cc52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:10:17.688951 28243 tablet_bootstrap.cc:492] T 91f3ece73a5a47839322fdabdc40cc52 P a2b8471244eb4bb4825f255d21e3cdba: Bootstrap starting.
I20260805 23:10:17.689803 28242 tablet_bootstrap.cc:492] T 91f3ece73a5a47839322fdabdc40cc52 P ee3ec60b51b44ca995c23c3908f540b5: Bootstrap starting.
I20260805 23:10:17.691339 28243 tablet_bootstrap.cc:654] T 91f3ece73a5a47839322fdabdc40cc52 P a2b8471244eb4bb4825f255d21e3cdba: Neither blocks nor log segments found. Creating new log.
I20260805 23:10:17.691560 28242 tablet_bootstrap.cc:654] T 91f3ece73a5a47839322fdabdc40cc52 P ee3ec60b51b44ca995c23c3908f540b5: Neither blocks nor log segments found. Creating new log.
I20260805 23:10:17.692129 28243 log.cc:824] T 91f3ece73a5a47839322fdabdc40cc52 P a2b8471244eb4bb4825f255d21e3cdba: Log is configured to *not* fsync() on all Append() calls
I20260805 23:10:17.692252 28242 log.cc:824] T 91f3ece73a5a47839322fdabdc40cc52 P ee3ec60b51b44ca995c23c3908f540b5: Log is configured to *not* fsync() on all Append() calls
I20260805 23:10:17.692874 28244 tablet_bootstrap.cc:492] T 91f3ece73a5a47839322fdabdc40cc52 P 3fa2896d2f224c69b1bb196eb4619a17: Bootstrap starting.
I20260805 23:10:17.694250 28243 tablet_bootstrap.cc:492] T 91f3ece73a5a47839322fdabdc40cc52 P a2b8471244eb4bb4825f255d21e3cdba: No bootstrap required, opened a new log
I20260805 23:10:17.694455 28243 ts_tablet_manager.cc:1397] T 91f3ece73a5a47839322fdabdc40cc52 P a2b8471244eb4bb4825f255d21e3cdba: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260805 23:10:17.694734 28242 tablet_bootstrap.cc:492] T 91f3ece73a5a47839322fdabdc40cc52 P ee3ec60b51b44ca995c23c3908f540b5: No bootstrap required, opened a new log
I20260805 23:10:17.694749 28244 tablet_bootstrap.cc:654] T 91f3ece73a5a47839322fdabdc40cc52 P 3fa2896d2f224c69b1bb196eb4619a17: Neither blocks nor log segments found. Creating new log.
I20260805 23:10:17.694902 28242 ts_tablet_manager.cc:1397] T 91f3ece73a5a47839322fdabdc40cc52 P ee3ec60b51b44ca995c23c3908f540b5: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260805 23:10:17.695586 28244 log.cc:824] T 91f3ece73a5a47839322fdabdc40cc52 P 3fa2896d2f224c69b1bb196eb4619a17: Log is configured to *not* fsync() on all Append() calls
I20260805 23:10:17.697310 28244 tablet_bootstrap.cc:492] T 91f3ece73a5a47839322fdabdc40cc52 P 3fa2896d2f224c69b1bb196eb4619a17: No bootstrap required, opened a new log
I20260805 23:10:17.697571 28244 ts_tablet_manager.cc:1397] T 91f3ece73a5a47839322fdabdc40cc52 P 3fa2896d2f224c69b1bb196eb4619a17: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260805 23:10:17.699246 28243 raft_consensus.cc:348] T 91f3ece73a5a47839322fdabdc40cc52 P a2b8471244eb4bb4825f255d21e3cdba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee3ec60b51b44ca995c23c3908f540b5" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38893 } } peers { permanent_uuid: "3fa2896d2f224c69b1bb196eb4619a17" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 42229 } } peers { permanent_uuid: "a2b8471244eb4bb4825f255d21e3cdba" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 34855 } }
I20260805 23:10:17.699246 28242 raft_consensus.cc:348] T 91f3ece73a5a47839322fdabdc40cc52 P ee3ec60b51b44ca995c23c3908f540b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee3ec60b51b44ca995c23c3908f540b5" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38893 } } peers { permanent_uuid: "3fa2896d2f224c69b1bb196eb4619a17" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 42229 } } peers { permanent_uuid: "a2b8471244eb4bb4825f255d21e3cdba" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 34855 } }
I20260805 23:10:17.699538 28242 raft_consensus.cc:374] T 91f3ece73a5a47839322fdabdc40cc52 P ee3ec60b51b44ca995c23c3908f540b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:10:17.699538 28243 raft_consensus.cc:374] T 91f3ece73a5a47839322fdabdc40cc52 P a2b8471244eb4bb4825f255d21e3cdba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:10:17.699602 28242 raft_consensus.cc:729] T 91f3ece73a5a47839322fdabdc40cc52 P ee3ec60b51b44ca995c23c3908f540b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee3ec60b51b44ca995c23c3908f540b5, State: Initialized, Role: FOLLOWER
I20260805 23:10:17.699604 28243 raft_consensus.cc:729] T 91f3ece73a5a47839322fdabdc40cc52 P a2b8471244eb4bb4825f255d21e3cdba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2b8471244eb4bb4825f255d21e3cdba, State: Initialized, Role: FOLLOWER
I20260805 23:10:17.700089 28242 consensus_queue.cc:260] T 91f3ece73a5a47839322fdabdc40cc52 P ee3ec60b51b44ca995c23c3908f540b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee3ec60b51b44ca995c23c3908f540b5" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38893 } } peers { permanent_uuid: "3fa2896d2f224c69b1bb196eb4619a17" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 42229 } } peers { permanent_uuid: "a2b8471244eb4bb4825f255d21e3cdba" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 34855 } }
I20260805 23:10:17.700078 28243 consensus_queue.cc:260] T 91f3ece73a5a47839322fdabdc40cc52 P a2b8471244eb4bb4825f255d21e3cdba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee3ec60b51b44ca995c23c3908f540b5" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38893 } } peers { permanent_uuid: "3fa2896d2f224c69b1bb196eb4619a17" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 42229 } } peers { permanent_uuid: "a2b8471244eb4bb4825f255d21e3cdba" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 34855 } }
I20260805 23:10:17.700795 27934 heartbeater.cc:503] Master 127.25.92.254:38325 was elected leader, sending a full tablet report...
I20260805 23:10:17.700713 28244 raft_consensus.cc:348] T 91f3ece73a5a47839322fdabdc40cc52 P 3fa2896d2f224c69b1bb196eb4619a17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee3ec60b51b44ca995c23c3908f540b5" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38893 } } peers { permanent_uuid: "3fa2896d2f224c69b1bb196eb4619a17" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 42229 } } peers { permanent_uuid: "a2b8471244eb4bb4825f255d21e3cdba" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 34855 } }
I20260805 23:10:17.700956 28244 raft_consensus.cc:374] T 91f3ece73a5a47839322fdabdc40cc52 P 3fa2896d2f224c69b1bb196eb4619a17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:10:17.701009 28244 raft_consensus.cc:729] T 91f3ece73a5a47839322fdabdc40cc52 P 3fa2896d2f224c69b1bb196eb4619a17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fa2896d2f224c69b1bb196eb4619a17, State: Initialized, Role: FOLLOWER
I20260805 23:10:17.701136 28242 ts_tablet_manager.cc:1428] T 91f3ece73a5a47839322fdabdc40cc52 P ee3ec60b51b44ca995c23c3908f540b5: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260805 23:10:17.701174 28243 ts_tablet_manager.cc:1428] T 91f3ece73a5a47839322fdabdc40cc52 P a2b8471244eb4bb4825f255d21e3cdba: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260805 23:10:17.701567 28218 heartbeater.cc:503] Master 127.25.92.254:38325 was elected leader, sending a full tablet report...
I20260805 23:10:17.701510 28244 consensus_queue.cc:260] T 91f3ece73a5a47839322fdabdc40cc52 P 3fa2896d2f224c69b1bb196eb4619a17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee3ec60b51b44ca995c23c3908f540b5" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38893 } } peers { permanent_uuid: "3fa2896d2f224c69b1bb196eb4619a17" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 42229 } } peers { permanent_uuid: "a2b8471244eb4bb4825f255d21e3cdba" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 34855 } }
I20260805 23:10:17.702332 28077 heartbeater.cc:503] Master 127.25.92.254:38325 was elected leader, sending a full tablet report...
I20260805 23:10:17.702466 28244 ts_tablet_manager.cc:1428] T 91f3ece73a5a47839322fdabdc40cc52 P 3fa2896d2f224c69b1bb196eb4619a17: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260805 23:10:17.776175 28250 raft_consensus.cc:482] T 91f3ece73a5a47839322fdabdc40cc52 P 3fa2896d2f224c69b1bb196eb4619a17 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 23:10:17.776419 28250 raft_consensus.cc:504] T 91f3ece73a5a47839322fdabdc40cc52 P 3fa2896d2f224c69b1bb196eb4619a17 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee3ec60b51b44ca995c23c3908f540b5" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38893 } } peers { permanent_uuid: "3fa2896d2f224c69b1bb196eb4619a17" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 42229 } } peers { permanent_uuid: "a2b8471244eb4bb4825f255d21e3cdba" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 34855 } }
I20260805 23:10:17.777818 28250 leader_election.cc:288] T 91f3ece73a5a47839322fdabdc40cc52 P 3fa2896d2f224c69b1bb196eb4619a17 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ee3ec60b51b44ca995c23c3908f540b5 (127.25.92.195:38893), a2b8471244eb4bb4825f255d21e3cdba (127.25.92.193:34855)
I20260805 23:10:17.782682 27969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:17.778113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.193:34855 (local address 127.25.92.194:56053)
0805 23:10:17.778238 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:17.778255 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:17.778358 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:17.778704 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:17.778708 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:17.778748 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:10:17.778891 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:17.778896 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:17.779851 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:17.779854 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:17.780987 (+ 1133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:17.780998 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:17.782162 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:17.782165 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:17.782306 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:17.782333 (+ 27us) client_negotiation.cc:769] Sending connection context
0805 23:10:17.782412 (+ 79us) client_negotiation.cc:240] Negotiation successful
0805 23:10:17.782496 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":42,"spinlock_wait_cycles":25216}
I20260805 23:10:17.782691 28240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:17.778256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.194:56053 (local address 127.25.92.193:34855)
0805 23:10:17.778448 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:17.778455 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:17.778486 (+ 31us) server_negotiation.cc:407] Connection header received
0805 23:10:17.778543 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:17.778545 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:17.778592 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:17.778669 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:10:17.779040 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:17.779729 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:17.781260 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:17.782032 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:17.782169 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:17.782269 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:17.782458 (+ 189us) server_negotiation.cc:299] Negotiation successful
0805 23:10:17.782555 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260805 23:10:17.782714 28237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:17.778253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.194:57883 (local address 127.25.92.195:38893)
0805 23:10:17.778408 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:17.778413 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:17.778738 (+ 325us) server_negotiation.cc:407] Connection header received
0805 23:10:17.778880 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:17.778882 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:17.778926 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:17.779051 (+ 125us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:10:17.779621 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:17.780228 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:17.781499 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:17.782169 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:17.782364 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:17.782423 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:17.782539 (+ 116us) server_negotiation.cc:299] Negotiation successful
0805 23:10:17.782605 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260805 23:10:17.782691 28251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:17.778125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.195:38893 (local address 127.25.92.194:57883)
0805 23:10:17.778617 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:17.778629 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:17.778755 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:17.779071 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:17.779074 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:17.779099 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:10:17.779438 (+ 339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:17.779443 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:17.780348 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:17.780352 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:17.781372 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:17.781380 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:17.782311 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:17.782314 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:17.782413 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:17.782431 (+ 18us) client_negotiation.cc:769] Sending connection context
0805 23:10:17.782495 (+ 64us) client_negotiation.cc:240] Negotiation successful
0805 23:10:17.782544 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":415,"mutex_wait_us":129,"spinlock_wait_cycles":8704,"thread_start_us":197,"threads_started":1}
I20260805 23:10:17.783385 28172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91f3ece73a5a47839322fdabdc40cc52" candidate_uuid: "3fa2896d2f224c69b1bb196eb4619a17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee3ec60b51b44ca995c23c3908f540b5" is_pre_election: true
I20260805 23:10:17.783382 27888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91f3ece73a5a47839322fdabdc40cc52" candidate_uuid: "3fa2896d2f224c69b1bb196eb4619a17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b8471244eb4bb4825f255d21e3cdba" is_pre_election: true
I20260805 23:10:17.783716 27888 raft_consensus.cc:2454] T 91f3ece73a5a47839322fdabdc40cc52 P a2b8471244eb4bb4825f255d21e3cdba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3fa2896d2f224c69b1bb196eb4619a17 in term 0.
I20260805 23:10:17.783716 28172 raft_consensus.cc:2454] T 91f3ece73a5a47839322fdabdc40cc52 P ee3ec60b51b44ca995c23c3908f540b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3fa2896d2f224c69b1bb196eb4619a17 in term 0.
I20260805 23:10:17.784291 27961 leader_election.cc:302] T 91f3ece73a5a47839322fdabdc40cc52 P 3fa2896d2f224c69b1bb196eb4619a17 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 3fa2896d2f224c69b1bb196eb4619a17, a2b8471244eb4bb4825f255d21e3cdba, ee3ec60b51b44ca995c23c3908f540b5; no voters:
I20260805 23:10:17.784586 28250 raft_consensus.cc:2781] T 91f3ece73a5a47839322fdabdc40cc52 P 3fa2896d2f224c69b1bb196eb4619a17 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260805 23:10:17.784698 28250 raft_consensus.cc:482] T 91f3ece73a5a47839322fdabdc40cc52 P 3fa2896d2f224c69b1bb196eb4619a17 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260805 23:10:17.784789 28250 raft_consensus.cc:3037] T 91f3ece73a5a47839322fdabdc40cc52 P 3fa2896d2f224c69b1bb196eb4619a17 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:10:17.785986 28250 raft_consensus.cc:504] T 91f3ece73a5a47839322fdabdc40cc52 P 3fa2896d2f224c69b1bb196eb4619a17 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee3ec60b51b44ca995c23c3908f540b5" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38893 } } peers { permanent_uuid: "3fa2896d2f224c69b1bb196eb4619a17" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 42229 } } peers { permanent_uuid: "a2b8471244eb4bb4825f255d21e3cdba" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 34855 } }
I20260805 23:10:17.786394 28250 leader_election.cc:288] T 91f3ece73a5a47839322fdabdc40cc52 P 3fa2896d2f224c69b1bb196eb4619a17 [CANDIDATE]: Term 1 election: Requested vote from peers ee3ec60b51b44ca995c23c3908f540b5 (127.25.92.195:38893), a2b8471244eb4bb4825f255d21e3cdba (127.25.92.193:34855)
I20260805 23:10:17.786830 28172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91f3ece73a5a47839322fdabdc40cc52" candidate_uuid: "3fa2896d2f224c69b1bb196eb4619a17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee3ec60b51b44ca995c23c3908f540b5"
I20260805 23:10:17.786830 27888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91f3ece73a5a47839322fdabdc40cc52" candidate_uuid: "3fa2896d2f224c69b1bb196eb4619a17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b8471244eb4bb4825f255d21e3cdba"
I20260805 23:10:17.786978 27888 raft_consensus.cc:3037] T 91f3ece73a5a47839322fdabdc40cc52 P a2b8471244eb4bb4825f255d21e3cdba [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:10:17.786978 28172 raft_consensus.cc:3037] T 91f3ece73a5a47839322fdabdc40cc52 P ee3ec60b51b44ca995c23c3908f540b5 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:10:17.788367 28172 raft_consensus.cc:2454] T 91f3ece73a5a47839322fdabdc40cc52 P ee3ec60b51b44ca995c23c3908f540b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fa2896d2f224c69b1bb196eb4619a17 in term 1.
I20260805 23:10:17.788367 27888 raft_consensus.cc:2454] T 91f3ece73a5a47839322fdabdc40cc52 P a2b8471244eb4bb4825f255d21e3cdba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fa2896d2f224c69b1bb196eb4619a17 in term 1.
I20260805 23:10:17.788789 27962 leader_election.cc:302] T 91f3ece73a5a47839322fdabdc40cc52 P 3fa2896d2f224c69b1bb196eb4619a17 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 3fa2896d2f224c69b1bb196eb4619a17, a2b8471244eb4bb4825f255d21e3cdba, ee3ec60b51b44ca995c23c3908f540b5; no voters:
I20260805 23:10:17.789101 28250 raft_consensus.cc:2781] T 91f3ece73a5a47839322fdabdc40cc52 P 3fa2896d2f224c69b1bb196eb4619a17 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:10:17.789361 28250 raft_consensus.cc:686] T 91f3ece73a5a47839322fdabdc40cc52 P 3fa2896d2f224c69b1bb196eb4619a17 [term 1 LEADER]: Becoming Leader. State: Replica: 3fa2896d2f224c69b1bb196eb4619a17, State: Running, Role: LEADER
I20260805 23:10:17.789757 28250 consensus_queue.cc:237] T 91f3ece73a5a47839322fdabdc40cc52 P 3fa2896d2f224c69b1bb196eb4619a17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee3ec60b51b44ca995c23c3908f540b5" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38893 } } peers { permanent_uuid: "3fa2896d2f224c69b1bb196eb4619a17" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 42229 } } peers { permanent_uuid: "a2b8471244eb4bb4825f255d21e3cdba" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 34855 } }
I20260805 23:10:17.793308 27727 catalog_manager.cc:5495] T 91f3ece73a5a47839322fdabdc40cc52 P 3fa2896d2f224c69b1bb196eb4619a17 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3fa2896d2f224c69b1bb196eb4619a17 (127.25.92.194). New cstate: current_term: 1 leader_uuid: "3fa2896d2f224c69b1bb196eb4619a17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee3ec60b51b44ca995c23c3908f540b5" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38893 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3fa2896d2f224c69b1bb196eb4619a17" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 42229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2b8471244eb4bb4825f255d21e3cdba" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 34855 } health_report { overall_health: UNKNOWN } } }
Aug 05 23:10:17 dist-test-slave-ss4n krb5kdc[27682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971416, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.254@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM
W20260805 23:10:17.862452 28219 tablet.cc:2369] T 91f3ece73a5a47839322fdabdc40cc52 P ee3ec60b51b44ca995c23c3908f540b5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260805 23:10:17.865319 27778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:17.855700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52136 (local address 127.25.92.254:38325)
0805 23:10:17.855916 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:17.855920 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:17.856093 (+ 173us) server_negotiation.cc:407] Connection header received
0805 23:10:17.856288 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:17.856291 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:17.856340 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:17.856440 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:17.857167 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:17.857879 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:17.858766 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:17.859014 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:17.859137 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:17.861837 (+ 2700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:17.861840 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:17.861842 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:17.861865 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:17.863523 (+ 1658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:17.864061 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:17.864063 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:17.864064 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:17.864138 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:17.864392 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:17.864394 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:17.864395 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:17.864574 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:17.864701 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:17.865006 (+ 305us) server_negotiation.cc:299] Negotiation successful
0805 23:10:17.865177 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":79,"server-negotiator.queue_time_us":109}
I20260805 23:10:17.865327 28259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:17.855562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:38325 (local address 127.0.0.1:52136)
0805 23:10:17.855990 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:17.856004 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:17.856166 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:17.856463 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:17.856466 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:17.856925 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:17.857036 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:17.857043 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:17.857995 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:17.858002 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:17.858635 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:17.858643 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:17.859142 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:17.859146 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:17.859260 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:17.859926 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:10:17.859939 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:17.861677 (+ 1738us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:10:17.863671 (+ 1994us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:17.863682 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:17.863688 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:17.863963 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:17.864224 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:17.864226 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:17.864228 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:17.864313 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:17.864724 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:10:17.864730 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:10:17.864848 (+ 118us) client_negotiation.cc:769] Sending connection context
0805 23:10:17.864953 (+ 105us) client_negotiation.cc:240] Negotiation successful
0805 23:10:17.865100 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"mutex_wait_us":6,"thread_start_us":143,"threads_started":1}
I20260805 23:10:17.870028 27727 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.25.92.254@KRBTEST.COM'} at 127.0.0.1:52136:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260805 23:10:17.870811 27727 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260805 23:10:17.871948 28078 tablet.cc:2369] T 91f3ece73a5a47839322fdabdc40cc52 P 3fa2896d2f224c69b1bb196eb4619a17: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260805 23:10:17.876157 28010 tablet_service.cc:1467] Processing CreateTablet for tablet 758f9a770bca4111b760884c93a2bc44 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=124ef3f939584ae59e7ad1570f723425]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260805 23:10:17.876225 28152 tablet_service.cc:1467] Processing CreateTablet for tablet 758f9a770bca4111b760884c93a2bc44 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=124ef3f939584ae59e7ad1570f723425]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260805 23:10:17.876080 27868 tablet_service.cc:1467] Processing CreateTablet for tablet 758f9a770bca4111b760884c93a2bc44 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=124ef3f939584ae59e7ad1570f723425]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260805 23:10:17.876883 27868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 758f9a770bca4111b760884c93a2bc44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:10:17.876883 28152 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 758f9a770bca4111b760884c93a2bc44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:10:17.876902 28010 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 758f9a770bca4111b760884c93a2bc44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:10:17.879607 28244 tablet_bootstrap.cc:492] T 758f9a770bca4111b760884c93a2bc44 P 3fa2896d2f224c69b1bb196eb4619a17: Bootstrap starting.
I20260805 23:10:17.879799 28242 tablet_bootstrap.cc:492] T 758f9a770bca4111b760884c93a2bc44 P ee3ec60b51b44ca995c23c3908f540b5: Bootstrap starting.
I20260805 23:10:17.880764 28244 tablet_bootstrap.cc:654] T 758f9a770bca4111b760884c93a2bc44 P 3fa2896d2f224c69b1bb196eb4619a17: Neither blocks nor log segments found. Creating new log.
I20260805 23:10:17.880798 28242 tablet_bootstrap.cc:654] T 758f9a770bca4111b760884c93a2bc44 P ee3ec60b51b44ca995c23c3908f540b5: Neither blocks nor log segments found. Creating new log.
I20260805 23:10:17.881413 28243 tablet_bootstrap.cc:492] T 758f9a770bca4111b760884c93a2bc44 P a2b8471244eb4bb4825f255d21e3cdba: Bootstrap starting.
I20260805 23:10:17.882155 28244 tablet_bootstrap.cc:492] T 758f9a770bca4111b760884c93a2bc44 P 3fa2896d2f224c69b1bb196eb4619a17: No bootstrap required, opened a new log
I20260805 23:10:17.882218 28242 tablet_bootstrap.cc:492] T 758f9a770bca4111b760884c93a2bc44 P ee3ec60b51b44ca995c23c3908f540b5: No bootstrap required, opened a new log
I20260805 23:10:17.882283 28244 ts_tablet_manager.cc:1397] T 758f9a770bca4111b760884c93a2bc44 P 3fa2896d2f224c69b1bb196eb4619a17: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260805 23:10:17.882316 28242 ts_tablet_manager.cc:1397] T 758f9a770bca4111b760884c93a2bc44 P ee3ec60b51b44ca995c23c3908f540b5: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260805 23:10:17.882593 28243 tablet_bootstrap.cc:654] T 758f9a770bca4111b760884c93a2bc44 P a2b8471244eb4bb4825f255d21e3cdba: Neither blocks nor log segments found. Creating new log.
I20260805 23:10:17.882820 28244 raft_consensus.cc:348] T 758f9a770bca4111b760884c93a2bc44 P 3fa2896d2f224c69b1bb196eb4619a17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2b8471244eb4bb4825f255d21e3cdba" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 34855 } } peers { permanent_uuid: "3fa2896d2f224c69b1bb196eb4619a17" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 42229 } } peers { permanent_uuid: "ee3ec60b51b44ca995c23c3908f540b5" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38893 } }
I20260805 23:10:17.882834 28242 raft_consensus.cc:348] T 758f9a770bca4111b760884c93a2bc44 P ee3ec60b51b44ca995c23c3908f540b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2b8471244eb4bb4825f255d21e3cdba" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 34855 } } peers { permanent_uuid: "3fa2896d2f224c69b1bb196eb4619a17" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 42229 } } peers { permanent_uuid: "ee3ec60b51b44ca995c23c3908f540b5" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38893 } }
I20260805 23:10:17.882952 28242 raft_consensus.cc:374] T 758f9a770bca4111b760884c93a2bc44 P ee3ec60b51b44ca995c23c3908f540b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:10:17.882947 28244 raft_consensus.cc:374] T 758f9a770bca4111b760884c93a2bc44 P 3fa2896d2f224c69b1bb196eb4619a17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:10:17.882985 28242 raft_consensus.cc:729] T 758f9a770bca4111b760884c93a2bc44 P ee3ec60b51b44ca995c23c3908f540b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee3ec60b51b44ca995c23c3908f540b5, State: Initialized, Role: FOLLOWER
I20260805 23:10:17.882990 28244 raft_consensus.cc:729] T 758f9a770bca4111b760884c93a2bc44 P 3fa2896d2f224c69b1bb196eb4619a17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fa2896d2f224c69b1bb196eb4619a17, State: Initialized, Role: FOLLOWER
I20260805 23:10:17.883136 28242 consensus_queue.cc:260] T 758f9a770bca4111b760884c93a2bc44 P ee3ec60b51b44ca995c23c3908f540b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2b8471244eb4bb4825f255d21e3cdba" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 34855 } } peers { permanent_uuid: "3fa2896d2f224c69b1bb196eb4619a17" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 42229 } } peers { permanent_uuid: "ee3ec60b51b44ca995c23c3908f540b5" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38893 } }
I20260805 23:10:17.883172 28244 consensus_queue.cc:260] T 758f9a770bca4111b760884c93a2bc44 P 3fa2896d2f224c69b1bb196eb4619a17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2b8471244eb4bb4825f255d21e3cdba" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 34855 } } peers { permanent_uuid: "3fa2896d2f224c69b1bb196eb4619a17" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 42229 } } peers { permanent_uuid: "ee3ec60b51b44ca995c23c3908f540b5" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38893 } }
I20260805 23:10:17.883514 28244 ts_tablet_manager.cc:1428] T 758f9a770bca4111b760884c93a2bc44 P 3fa2896d2f224c69b1bb196eb4619a17: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260805 23:10:17.883425 28249 tablet_replica.cc:439] T 758f9a770bca4111b760884c93a2bc44 P ee3ec60b51b44ca995c23c3908f540b5: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2b8471244eb4bb4825f255d21e3cdba" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 34855 } } peers { permanent_uuid: "3fa2896d2f224c69b1bb196eb4619a17" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 42229 } } peers { permanent_uuid: "ee3ec60b51b44ca995c23c3908f540b5" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38893 } } }
I20260805 23:10:17.883584 28249 tablet_replica.cc:442] T 758f9a770bca4111b760884c93a2bc44 P ee3ec60b51b44ca995c23c3908f540b5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260805 23:10:17.883914 28250 tablet_replica.cc:439] T 758f9a770bca4111b760884c93a2bc44 P 3fa2896d2f224c69b1bb196eb4619a17: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2b8471244eb4bb4825f255d21e3cdba" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 34855 } } peers { permanent_uuid: "3fa2896d2f224c69b1bb196eb4619a17" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 42229 } } peers { permanent_uuid: "ee3ec60b51b44ca995c23c3908f540b5" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38893 } } }
I20260805 23:10:17.884059 28250 tablet_replica.cc:442] T 758f9a770bca4111b760884c93a2bc44 P 3fa2896d2f224c69b1bb196eb4619a17: This TxnStatusTablet replica's current role is: FOLLOWER
I20260805 23:10:17.884145 28243 tablet_bootstrap.cc:492] T 758f9a770bca4111b760884c93a2bc44 P a2b8471244eb4bb4825f255d21e3cdba: No bootstrap required, opened a new log
I20260805 23:10:17.884229 28243 ts_tablet_manager.cc:1397] T 758f9a770bca4111b760884c93a2bc44 P a2b8471244eb4bb4825f255d21e3cdba: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260805 23:10:17.883440 28242 ts_tablet_manager.cc:1428] T 758f9a770bca4111b760884c93a2bc44 P ee3ec60b51b44ca995c23c3908f540b5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260805 23:10:17.884675 28243 raft_consensus.cc:348] T 758f9a770bca4111b760884c93a2bc44 P a2b8471244eb4bb4825f255d21e3cdba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2b8471244eb4bb4825f255d21e3cdba" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 34855 } } peers { permanent_uuid: "3fa2896d2f224c69b1bb196eb4619a17" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 42229 } } peers { permanent_uuid: "ee3ec60b51b44ca995c23c3908f540b5" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38893 } }
I20260805 23:10:17.884799 28243 raft_consensus.cc:374] T 758f9a770bca4111b760884c93a2bc44 P a2b8471244eb4bb4825f255d21e3cdba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:10:17.884835 28243 raft_consensus.cc:729] T 758f9a770bca4111b760884c93a2bc44 P a2b8471244eb4bb4825f255d21e3cdba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2b8471244eb4bb4825f255d21e3cdba, State: Initialized, Role: FOLLOWER
I20260805 23:10:17.884970 28243 consensus_queue.cc:260] T 758f9a770bca4111b760884c93a2bc44 P a2b8471244eb4bb4825f255d21e3cdba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2b8471244eb4bb4825f255d21e3cdba" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 34855 } } peers { permanent_uuid: "3fa2896d2f224c69b1bb196eb4619a17" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 42229 } } peers { permanent_uuid: "ee3ec60b51b44ca995c23c3908f540b5" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38893 } }
I20260805 23:10:17.885262 28243 ts_tablet_manager.cc:1428] T 758f9a770bca4111b760884c93a2bc44 P a2b8471244eb4bb4825f255d21e3cdba: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260805 23:10:17.885321 28248 tablet_replica.cc:439] T 758f9a770bca4111b760884c93a2bc44 P a2b8471244eb4bb4825f255d21e3cdba: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2b8471244eb4bb4825f255d21e3cdba" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 34855 } } peers { permanent_uuid: "3fa2896d2f224c69b1bb196eb4619a17" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 42229 } } peers { permanent_uuid: "ee3ec60b51b44ca995c23c3908f540b5" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38893 } } }
I20260805 23:10:17.885519 28248 tablet_replica.cc:442] T 758f9a770bca4111b760884c93a2bc44 P a2b8471244eb4bb4825f255d21e3cdba: This TxnStatusTablet replica's current role is: FOLLOWER
W20260805 23:10:17.890014 27935 tablet.cc:2369] T 758f9a770bca4111b760884c93a2bc44 P a2b8471244eb4bb4825f255d21e3cdba: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260805 23:10:17.929948 28248 raft_consensus.cc:482] T 758f9a770bca4111b760884c93a2bc44 P a2b8471244eb4bb4825f255d21e3cdba [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 23:10:17.930147 28248 raft_consensus.cc:504] T 758f9a770bca4111b760884c93a2bc44 P a2b8471244eb4bb4825f255d21e3cdba [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2b8471244eb4bb4825f255d21e3cdba" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 34855 } } peers { permanent_uuid: "3fa2896d2f224c69b1bb196eb4619a17" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 42229 } } peers { permanent_uuid: "ee3ec60b51b44ca995c23c3908f540b5" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38893 } }
I20260805 23:10:17.931277 28248 leader_election.cc:288] T 758f9a770bca4111b760884c93a2bc44 P a2b8471244eb4bb4825f255d21e3cdba [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3fa2896d2f224c69b1bb196eb4619a17 (127.25.92.194:42229), ee3ec60b51b44ca995c23c3908f540b5 (127.25.92.195:38893)
I20260805 23:10:17.937304 28241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:17.931788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.193:43813 (local address 127.25.92.194:42229)
0805 23:10:17.931965 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:17.931969 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:17.932173 (+ 204us) server_negotiation.cc:407] Connection header received
0805 23:10:17.932410 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:17.932417 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:17.932478 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:17.932602 (+ 124us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:10:17.932966 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:17.933988 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:17.935897 (+ 1909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:17.936497 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:17.936640 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:17.936698 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:17.937050 (+ 352us) server_negotiation.cc:299] Negotiation successful
0805 23:10:17.937169 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260805 23:10:17.937345 28237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:17.932329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.193:37049 (local address 127.25.92.195:38893)
0805 23:10:17.932504 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:17.932508 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:17.932727 (+ 219us) server_negotiation.cc:407] Connection header received
0805 23:10:17.932858 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:17.932864 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:17.932932 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:17.933010 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:10:17.934030 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:17.934624 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:17.936099 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:17.936725 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:17.936832 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:17.936866 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:17.937161 (+ 295us) server_negotiation.cc:299] Negotiation successful
0805 23:10:17.937239 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260805 23:10:17.937551 28264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:17.932208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.195:38893 (local address 127.25.92.193:37049)
0805 23:10:17.932620 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:17.932636 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:17.932751 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:17.933056 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:17.933058 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:17.933079 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:10:17.933825 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:17.933837 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:17.934741 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:17.934745 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:17.935941 (+ 1196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:17.935950 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:17.936859 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:17.936863 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:17.937005 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:17.937049 (+ 44us) client_negotiation.cc:769] Sending connection context
0805 23:10:17.937127 (+ 78us) client_negotiation.cc:240] Negotiation successful
0805 23:10:17.937184 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"mutex_wait_us":27,"thread_start_us":113,"threads_started":1}
I20260805 23:10:17.937968 28263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:17.931707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.194:42229 (local address 127.25.92.193:43813)
0805 23:10:17.932048 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:17.932066 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:17.932199 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:17.932612 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:17.932615 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:17.932634 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:10:17.932827 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:17.932836 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:17.934086 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:17.934092 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:17.935527 (+ 1435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:17.935539 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:17.936689 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:17.936692 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:17.936859 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:17.936891 (+ 32us) client_negotiation.cc:769] Sending connection context
0805 23:10:17.936991 (+ 100us) client_negotiation.cc:240] Negotiation successful
0805 23:10:17.937136 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"mutex_wait_us":35,"thread_start_us":153,"threads_started":1}
I20260805 23:10:17.938136 28172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "758f9a770bca4111b760884c93a2bc44" candidate_uuid: "a2b8471244eb4bb4825f255d21e3cdba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee3ec60b51b44ca995c23c3908f540b5" is_pre_election: true
I20260805 23:10:17.938301 28172 raft_consensus.cc:2454] T 758f9a770bca4111b760884c93a2bc44 P ee3ec60b51b44ca995c23c3908f540b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2b8471244eb4bb4825f255d21e3cdba in term 0.
I20260805 23:10:17.938804 27818 leader_election.cc:302] T 758f9a770bca4111b760884c93a2bc44 P a2b8471244eb4bb4825f255d21e3cdba [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2b8471244eb4bb4825f255d21e3cdba, ee3ec60b51b44ca995c23c3908f540b5; no voters:
I20260805 23:10:17.939126 28248 raft_consensus.cc:2781] T 758f9a770bca4111b760884c93a2bc44 P a2b8471244eb4bb4825f255d21e3cdba [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260805 23:10:17.939244 28248 raft_consensus.cc:482] T 758f9a770bca4111b760884c93a2bc44 P a2b8471244eb4bb4825f255d21e3cdba [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260805 23:10:17.939316 28248 raft_consensus.cc:3037] T 758f9a770bca4111b760884c93a2bc44 P a2b8471244eb4bb4825f255d21e3cdba [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:10:17.939571 28030 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "758f9a770bca4111b760884c93a2bc44" candidate_uuid: "a2b8471244eb4bb4825f255d21e3cdba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fa2896d2f224c69b1bb196eb4619a17" is_pre_election: true
I20260805 23:10:17.939805 28030 raft_consensus.cc:2454] T 758f9a770bca4111b760884c93a2bc44 P 3fa2896d2f224c69b1bb196eb4619a17 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2b8471244eb4bb4825f255d21e3cdba in term 0.
I20260805 23:10:17.940744 28248 raft_consensus.cc:504] T 758f9a770bca4111b760884c93a2bc44 P a2b8471244eb4bb4825f255d21e3cdba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2b8471244eb4bb4825f255d21e3cdba" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 34855 } } peers { permanent_uuid: "3fa2896d2f224c69b1bb196eb4619a17" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 42229 } } peers { permanent_uuid: "ee3ec60b51b44ca995c23c3908f540b5" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38893 } }
I20260805 23:10:17.941219 28248 leader_election.cc:288] T 758f9a770bca4111b760884c93a2bc44 P a2b8471244eb4bb4825f255d21e3cdba [CANDIDATE]: Term 1 election: Requested vote from peers 3fa2896d2f224c69b1bb196eb4619a17 (127.25.92.194:42229), ee3ec60b51b44ca995c23c3908f540b5 (127.25.92.195:38893)
I20260805 23:10:17.941655 28030 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "758f9a770bca4111b760884c93a2bc44" candidate_uuid: "a2b8471244eb4bb4825f255d21e3cdba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fa2896d2f224c69b1bb196eb4619a17"
I20260805 23:10:17.941779 28172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "758f9a770bca4111b760884c93a2bc44" candidate_uuid: "a2b8471244eb4bb4825f255d21e3cdba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee3ec60b51b44ca995c23c3908f540b5"
I20260805 23:10:17.941835 28030 raft_consensus.cc:3037] T 758f9a770bca4111b760884c93a2bc44 P 3fa2896d2f224c69b1bb196eb4619a17 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:10:17.941936 28172 raft_consensus.cc:3037] T 758f9a770bca4111b760884c93a2bc44 P ee3ec60b51b44ca995c23c3908f540b5 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:10:17.943097 28030 raft_consensus.cc:2454] T 758f9a770bca4111b760884c93a2bc44 P 3fa2896d2f224c69b1bb196eb4619a17 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2b8471244eb4bb4825f255d21e3cdba in term 1.
I20260805 23:10:17.943157 28172 raft_consensus.cc:2454] T 758f9a770bca4111b760884c93a2bc44 P ee3ec60b51b44ca995c23c3908f540b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2b8471244eb4bb4825f255d21e3cdba in term 1.
I20260805 23:10:17.943477 27818 leader_election.cc:302] T 758f9a770bca4111b760884c93a2bc44 P a2b8471244eb4bb4825f255d21e3cdba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2b8471244eb4bb4825f255d21e3cdba, ee3ec60b51b44ca995c23c3908f540b5; no voters:
I20260805 23:10:17.943697 28248 raft_consensus.cc:2781] T 758f9a770bca4111b760884c93a2bc44 P a2b8471244eb4bb4825f255d21e3cdba [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:10:17.943987 28248 raft_consensus.cc:686] T 758f9a770bca4111b760884c93a2bc44 P a2b8471244eb4bb4825f255d21e3cdba [term 1 LEADER]: Becoming Leader. State: Replica: a2b8471244eb4bb4825f255d21e3cdba, State: Running, Role: LEADER
I20260805 23:10:17.944446 28248 consensus_queue.cc:237] T 758f9a770bca4111b760884c93a2bc44 P a2b8471244eb4bb4825f255d21e3cdba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2b8471244eb4bb4825f255d21e3cdba" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 34855 } } peers { permanent_uuid: "3fa2896d2f224c69b1bb196eb4619a17" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 42229 } } peers { permanent_uuid: "ee3ec60b51b44ca995c23c3908f540b5" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38893 } }
I20260805 23:10:17.947417 28265 tablet_replica.cc:439] T 758f9a770bca4111b760884c93a2bc44 P a2b8471244eb4bb4825f255d21e3cdba: TxnStatusTablet state changed. Reason: New leader a2b8471244eb4bb4825f255d21e3cdba. Latest consensus state: current_term: 1 leader_uuid: "a2b8471244eb4bb4825f255d21e3cdba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2b8471244eb4bb4825f255d21e3cdba" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 34855 } } peers { permanent_uuid: "3fa2896d2f224c69b1bb196eb4619a17" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 42229 } } peers { permanent_uuid: "ee3ec60b51b44ca995c23c3908f540b5" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38893 } } }
I20260805 23:10:17.947568 28265 tablet_replica.cc:442] T 758f9a770bca4111b760884c93a2bc44 P a2b8471244eb4bb4825f255d21e3cdba: This TxnStatusTablet replica's current role is: LEADER
I20260805 23:10:17.948194 28267 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260805 23:10:17.948453 28267 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260805 23:10:17.948956 27728 catalog_manager.cc:5495] T 758f9a770bca4111b760884c93a2bc44 P a2b8471244eb4bb4825f255d21e3cdba reported cstate change: term changed from 0 to 1, leader changed from <none> to a2b8471244eb4bb4825f255d21e3cdba (127.25.92.193). New cstate: current_term: 1 leader_uuid: "a2b8471244eb4bb4825f255d21e3cdba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2b8471244eb4bb4825f255d21e3cdba" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 34855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3fa2896d2f224c69b1bb196eb4619a17" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 42229 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee3ec60b51b44ca995c23c3908f540b5" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38893 } health_report { overall_health: UNKNOWN } } }
I20260805 23:10:17.974238 28259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:17.970415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.193:34855 (local address 127.0.0.1:49552)
0805 23:10:17.970572 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:17.970591 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:17.970706 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:17.971051 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:17.971054 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:17.971062 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0805 23:10:17.971185 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:17.971190 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:17.971930 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:17.971935 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:17.972541 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:17.972549 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:17.973013 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:17.973017 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:17.973124 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:17.973173 (+ 49us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0805 23:10:17.973928 (+ 755us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0805 23:10:17.973945 (+ 17us) client_negotiation.cc:769] Sending connection context
0805 23:10:17.974016 (+ 71us) client_negotiation.cc:240] Negotiation successful
0805 23:10:17.974118 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260805 23:10:17.974238 28240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:17.970544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49552 (local address 127.25.92.193:34855)
0805 23:10:17.970685 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:17.970690 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:17.970704 (+ 14us) server_negotiation.cc:407] Connection header received
0805 23:10:17.970873 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:17.970876 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:17.970937 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:17.971036 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 23:10:17.971294 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:17.971817 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:17.972666 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:17.972905 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:17.973013 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:17.973331 (+ 318us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 23:10:17.973827 (+ 496us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 23:10:17.973957 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:17.974027 (+ 70us) server_negotiation.cc:299] Negotiation successful
0805 23:10:17.974085 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260805 23:10:17.979943 28030 raft_consensus.cc:1261] T 758f9a770bca4111b760884c93a2bc44 P 3fa2896d2f224c69b1bb196eb4619a17 [term 1 FOLLOWER]: Refusing update from remote peer a2b8471244eb4bb4825f255d21e3cdba: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:10:17.980106 28172 raft_consensus.cc:1261] T 758f9a770bca4111b760884c93a2bc44 P ee3ec60b51b44ca995c23c3908f540b5 [term 1 FOLLOWER]: Refusing update from remote peer a2b8471244eb4bb4825f255d21e3cdba: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:10:17.980636 28265 consensus_queue.cc:1035] T 758f9a770bca4111b760884c93a2bc44 P a2b8471244eb4bb4825f255d21e3cdba [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fa2896d2f224c69b1bb196eb4619a17" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:10:17.980829 28248 consensus_queue.cc:1035] T 758f9a770bca4111b760884c93a2bc44 P a2b8471244eb4bb4825f255d21e3cdba [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee3ec60b51b44ca995c23c3908f540b5" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38893 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:10:17.984028 28250 tablet_replica.cc:439] T 758f9a770bca4111b760884c93a2bc44 P 3fa2896d2f224c69b1bb196eb4619a17: TxnStatusTablet state changed. Reason: New leader a2b8471244eb4bb4825f255d21e3cdba. Latest consensus state: current_term: 1 leader_uuid: "a2b8471244eb4bb4825f255d21e3cdba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2b8471244eb4bb4825f255d21e3cdba" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 34855 } } peers { permanent_uuid: "3fa2896d2f224c69b1bb196eb4619a17" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 42229 } } peers { permanent_uuid: "ee3ec60b51b44ca995c23c3908f540b5" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38893 } } }
I20260805 23:10:17.984150 28250 tablet_replica.cc:442] T 758f9a770bca4111b760884c93a2bc44 P 3fa2896d2f224c69b1bb196eb4619a17: This TxnStatusTablet replica's current role is: FOLLOWER
I20260805 23:10:17.984949 28249 tablet_replica.cc:439] T 758f9a770bca4111b760884c93a2bc44 P ee3ec60b51b44ca995c23c3908f540b5: TxnStatusTablet state changed. Reason: New leader a2b8471244eb4bb4825f255d21e3cdba. Latest consensus state: current_term: 1 leader_uuid: "a2b8471244eb4bb4825f255d21e3cdba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2b8471244eb4bb4825f255d21e3cdba" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 34855 } } peers { permanent_uuid: "3fa2896d2f224c69b1bb196eb4619a17" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 42229 } } peers { permanent_uuid: "ee3ec60b51b44ca995c23c3908f540b5" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38893 } } }
I20260805 23:10:17.985091 28249 tablet_replica.cc:442] T 758f9a770bca4111b760884c93a2bc44 P ee3ec60b51b44ca995c23c3908f540b5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260805 23:10:17.988719 28248 tablet_replica.cc:439] T 758f9a770bca4111b760884c93a2bc44 P a2b8471244eb4bb4825f255d21e3cdba: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a2b8471244eb4bb4825f255d21e3cdba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2b8471244eb4bb4825f255d21e3cdba" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 34855 } } peers { permanent_uuid: "3fa2896d2f224c69b1bb196eb4619a17" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 42229 } } peers { permanent_uuid: "ee3ec60b51b44ca995c23c3908f540b5" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38893 } } }
I20260805 23:10:17.988842 28248 tablet_replica.cc:442] T 758f9a770bca4111b760884c93a2bc44 P a2b8471244eb4bb4825f255d21e3cdba: This TxnStatusTablet replica's current role is: LEADER
I20260805 23:10:17.988940 28248 tablet_replica.cc:439] T 758f9a770bca4111b760884c93a2bc44 P a2b8471244eb4bb4825f255d21e3cdba: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a2b8471244eb4bb4825f255d21e3cdba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2b8471244eb4bb4825f255d21e3cdba" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 34855 } } peers { permanent_uuid: "3fa2896d2f224c69b1bb196eb4619a17" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 42229 } } peers { permanent_uuid: "ee3ec60b51b44ca995c23c3908f540b5" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38893 } } }
I20260805 23:10:17.989012 28248 tablet_replica.cc:442] T 758f9a770bca4111b760884c93a2bc44 P a2b8471244eb4bb4825f255d21e3cdba: This TxnStatusTablet replica's current role is: LEADER
I20260805 23:10:17.990832 28250 tablet_replica.cc:439] T 758f9a770bca4111b760884c93a2bc44 P 3fa2896d2f224c69b1bb196eb4619a17: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a2b8471244eb4bb4825f255d21e3cdba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2b8471244eb4bb4825f255d21e3cdba" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 34855 } } peers { permanent_uuid: "3fa2896d2f224c69b1bb196eb4619a17" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 42229 } } peers { permanent_uuid: "ee3ec60b51b44ca995c23c3908f540b5" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38893 } } }
I20260805 23:10:17.990949 28250 tablet_replica.cc:442] T 758f9a770bca4111b760884c93a2bc44 P 3fa2896d2f224c69b1bb196eb4619a17: This TxnStatusTablet replica's current role is: FOLLOWER
I20260805 23:10:17.991235 28272 mvcc.cc:205] Tried to move back new op lower bound from 7315338928038985728 to 7315338927903596544. Current Snapshot: MvccSnapshot[applied={T|T < 7315338928038985728}]
I20260805 23:10:17.993642 28249 tablet_replica.cc:439] T 758f9a770bca4111b760884c93a2bc44 P ee3ec60b51b44ca995c23c3908f540b5: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a2b8471244eb4bb4825f255d21e3cdba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2b8471244eb4bb4825f255d21e3cdba" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 34855 } } peers { permanent_uuid: "3fa2896d2f224c69b1bb196eb4619a17" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 42229 } } peers { permanent_uuid: "ee3ec60b51b44ca995c23c3908f540b5" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38893 } } }
I20260805 23:10:17.993788 28249 tablet_replica.cc:442] T 758f9a770bca4111b760884c93a2bc44 P ee3ec60b51b44ca995c23c3908f540b5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260805 23:10:18.002573 28241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:17.998509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33224 (local address 127.25.92.194:42229)
0805 23:10:17.998623 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:17.998629 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:17.998677 (+ 48us) server_negotiation.cc:407] Connection header received
0805 23:10:17.998891 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:17.998893 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:17.998948 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:17.999060 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 23:10:17.999311 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:17.999810 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:18.000884 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:18.001176 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:18.001287 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:18.001619 (+ 332us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 23:10:18.002165 (+ 546us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 23:10:18.002307 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:18.002374 (+ 67us) server_negotiation.cc:299] Negotiation successful
0805 23:10:18.002429 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260805 23:10:18.014189 28283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:18.010536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.193:34855 (local address 127.25.92.194:49441)
0805 23:10:18.010882 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:18.010896 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:18.011029 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:18.011304 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:18.011307 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:18.011315 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0805 23:10:18.011446 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:18.011451 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:18.012197 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:18.012201 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:18.012844 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:18.012852 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:18.013320 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:18.013324 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:18.013421 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:18.013474 (+ 53us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0805 23:10:18.013869 (+ 395us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0805 23:10:18.013879 (+ 10us) client_negotiation.cc:769] Sending connection context
0805 23:10:18.013946 (+ 67us) client_negotiation.cc:240] Negotiation successful
0805 23:10:18.014031 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":114,"threads_started":1}
I20260805 23:10:18.014163 28240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:18.010708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.194:49441 (local address 127.25.92.193:34855)
0805 23:10:18.010854 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:18.010859 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:18.010983 (+ 124us) server_negotiation.cc:407] Connection header received
0805 23:10:18.011147 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:18.011151 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:18.011207 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:18.011291 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 23:10:18.011551 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:18.012080 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:18.012966 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:18.013204 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:18.013318 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:18.013703 (+ 385us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 23:10:18.013779 (+ 76us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 23:10:18.013864 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:18.013956 (+ 92us) server_negotiation.cc:299] Negotiation successful
0805 23:10:18.014015 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260805 23:10:18.022442 28172 raft_consensus.cc:1261] T 91f3ece73a5a47839322fdabdc40cc52 P ee3ec60b51b44ca995c23c3908f540b5 [term 1 FOLLOWER]: Refusing update from remote peer 3fa2896d2f224c69b1bb196eb4619a17: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:10:18.022815 27888 raft_consensus.cc:1261] T 91f3ece73a5a47839322fdabdc40cc52 P a2b8471244eb4bb4825f255d21e3cdba [term 1 FOLLOWER]: Refusing update from remote peer 3fa2896d2f224c69b1bb196eb4619a17: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:10:18.022950 28250 consensus_queue.cc:1035] T 91f3ece73a5a47839322fdabdc40cc52 P 3fa2896d2f224c69b1bb196eb4619a17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee3ec60b51b44ca995c23c3908f540b5" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38893 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:10:18.023375 28250 consensus_queue.cc:1035] T 91f3ece73a5a47839322fdabdc40cc52 P 3fa2896d2f224c69b1bb196eb4619a17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b8471244eb4bb4825f255d21e3cdba" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:10:18.031186 28270 mvcc.cc:205] Tried to move back new op lower bound from 7315338928215117824 to 7315338927269548032. Current Snapshot: MvccSnapshot[applied={T|T < 7315338928215117824}]
I20260805 23:10:18.047452 28241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:18.043217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.193:48609 (local address 127.25.92.194:42229)
0805 23:10:18.043360 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:18.043365 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:18.043576 (+ 211us) server_negotiation.cc:407] Connection header received
0805 23:10:18.043765 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:18.043768 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:18.043820 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:18.043902 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 23:10:18.044153 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:18.045013 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:18.046286 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:18.046504 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:18.046625 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:18.046938 (+ 313us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 23:10:18.047022 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 23:10:18.047094 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:18.047243 (+ 149us) server_negotiation.cc:299] Negotiation successful
0805 23:10:18.047307 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260805 23:10:18.047474 28287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:18.043132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.194:42229 (local address 127.25.92.193:48609)
0805 23:10:18.043460 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:18.043475 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:18.043624 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:18.043927 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:18.043930 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:18.043938 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0805 23:10:18.044045 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:18.044052 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:18.045152 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:18.045155 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:18.046162 (+ 1007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:18.046174 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:18.046628 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:18.046632 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:18.046722 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:18.046778 (+ 56us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0805 23:10:18.047107 (+ 329us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0805 23:10:18.047116 (+ 9us) client_negotiation.cc:769] Sending connection context
0805 23:10:18.047218 (+ 102us) client_negotiation.cc:240] Negotiation successful
0805 23:10:18.047309 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":112,"threads_started":1}
I20260805 23:10:18.077085 27724 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52132:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:10:18.077306 27724 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52132:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:10:18.080467 27724 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ea219c710c774b9d9e270e6fad216a05: Sending DeleteTablet for 3 replicas of tablet 91f3ece73a5a47839322fdabdc40cc52
I20260805 23:10:18.081547 28152 tablet_service.cc:1514] Processing DeleteTablet for tablet 91f3ece73a5a47839322fdabdc40cc52 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:10:18 UTC) from {username='kudu', principal='kudu/127.25.92.254@KRBTEST.COM'} at 127.0.0.1:46668
I20260805 23:10:18.082139 28291 tablet_replica.cc:330] T 91f3ece73a5a47839322fdabdc40cc52 P ee3ec60b51b44ca995c23c3908f540b5: stopping tablet replica
I20260805 23:10:18.082351 28010 tablet_service.cc:1514] Processing DeleteTablet for tablet 91f3ece73a5a47839322fdabdc40cc52 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:10:18 UTC) from {username='kudu', principal='kudu/127.25.92.254@KRBTEST.COM'} at 127.0.0.1:33210
I20260805 23:10:18.082788 28291 raft_consensus.cc:2229] T 91f3ece73a5a47839322fdabdc40cc52 P ee3ec60b51b44ca995c23c3908f540b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 23:10:18.082911 27868 tablet_service.cc:1514] Processing DeleteTablet for tablet 91f3ece73a5a47839322fdabdc40cc52 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:10:18 UTC) from {username='kudu', principal='kudu/127.25.92.254@KRBTEST.COM'} at 127.0.0.1:49550
I20260805 23:10:18.083160 28291 raft_consensus.cc:2258] T 91f3ece73a5a47839322fdabdc40cc52 P ee3ec60b51b44ca995c23c3908f540b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:10:18.084444 27724 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52132:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260805 23:10:18.084971 27724 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260805 23:10:18.085183 28291 ts_tablet_manager.cc:1905] T 91f3ece73a5a47839322fdabdc40cc52 P ee3ec60b51b44ca995c23c3908f540b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:10:18.088433 28291 ts_tablet_manager.cc:1918] T 91f3ece73a5a47839322fdabdc40cc52 P ee3ec60b51b44ca995c23c3908f540b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260805 23:10:18.088641 28291 log.cc:1196] T 91f3ece73a5a47839322fdabdc40cc52 P ee3ec60b51b44ca995c23c3908f540b5: Deleting WAL directory at /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/minicluster-data/ts-2/wal/wals/91f3ece73a5a47839322fdabdc40cc52
I20260805 23:10:18.089035 28291 ts_tablet_manager.cc:1939] T 91f3ece73a5a47839322fdabdc40cc52 P ee3ec60b51b44ca995c23c3908f540b5: Deleting consensus metadata
I20260805 23:10:18.090824 27711 catalog_manager.cc:4841] TS ee3ec60b51b44ca995c23c3908f540b5 (127.25.92.195:38893): tablet 91f3ece73a5a47839322fdabdc40cc52 (table test-table [id=69d9b3bdd65648418911b2ec3e39bd84]) successfully deleted
I20260805 23:10:18.091048 27868 tablet_service.cc:1467] Processing CreateTablet for tablet 0a5661662ac2408ba029dfb031dac83c (DEFAULT_TABLE table=test-table [id=0578120a5b864013a5243eeeaeaed5ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:10:18.091338 27868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a5661662ac2408ba029dfb031dac83c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:10:18.091389 28152 tablet_service.cc:1467] Processing CreateTablet for tablet 0a5661662ac2408ba029dfb031dac83c (DEFAULT_TABLE table=test-table [id=0578120a5b864013a5243eeeaeaed5ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:10:18.091665 28152 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a5661662ac2408ba029dfb031dac83c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:10:18.093863 28243 tablet_bootstrap.cc:492] T 0a5661662ac2408ba029dfb031dac83c P a2b8471244eb4bb4825f255d21e3cdba: Bootstrap starting.
I20260805 23:10:18.093983 28010 tablet_service.cc:1467] Processing CreateTablet for tablet 0a5661662ac2408ba029dfb031dac83c (DEFAULT_TABLE table=test-table [id=0578120a5b864013a5243eeeaeaed5ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:10:18.094290 28010 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a5661662ac2408ba029dfb031dac83c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:10:18.095587 28243 tablet_bootstrap.cc:654] T 0a5661662ac2408ba029dfb031dac83c P a2b8471244eb4bb4825f255d21e3cdba: Neither blocks nor log segments found. Creating new log.
I20260805 23:10:18.096977 28243 tablet_bootstrap.cc:492] T 0a5661662ac2408ba029dfb031dac83c P a2b8471244eb4bb4825f255d21e3cdba: No bootstrap required, opened a new log
I20260805 23:10:18.097069 28243 ts_tablet_manager.cc:1397] T 0a5661662ac2408ba029dfb031dac83c P a2b8471244eb4bb4825f255d21e3cdba: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.001s
I20260805 23:10:18.097741 28243 raft_consensus.cc:348] T 0a5661662ac2408ba029dfb031dac83c P a2b8471244eb4bb4825f255d21e3cdba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2b8471244eb4bb4825f255d21e3cdba" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 34855 } } peers { permanent_uuid: "3fa2896d2f224c69b1bb196eb4619a17" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 42229 } } peers { permanent_uuid: "ee3ec60b51b44ca995c23c3908f540b5" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38893 } }
I20260805 23:10:18.097952 28243 raft_consensus.cc:374] T 0a5661662ac2408ba029dfb031dac83c P a2b8471244eb4bb4825f255d21e3cdba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:10:18.098024 28243 raft_consensus.cc:729] T 0a5661662ac2408ba029dfb031dac83c P a2b8471244eb4bb4825f255d21e3cdba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2b8471244eb4bb4825f255d21e3cdba, State: Initialized, Role: FOLLOWER
I20260805 23:10:18.099857 28244 tablet_bootstrap.cc:492] T 0a5661662ac2408ba029dfb031dac83c P 3fa2896d2f224c69b1bb196eb4619a17: Bootstrap starting.
I20260805 23:10:18.101109 28244 tablet_bootstrap.cc:654] T 0a5661662ac2408ba029dfb031dac83c P 3fa2896d2f224c69b1bb196eb4619a17: Neither blocks nor log segments found. Creating new log.
I20260805 23:10:18.098240 28243 consensus_queue.cc:260] T 0a5661662ac2408ba029dfb031dac83c P a2b8471244eb4bb4825f255d21e3cdba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2b8471244eb4bb4825f255d21e3cdba" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 34855 } } peers { permanent_uuid: "3fa2896d2f224c69b1bb196eb4619a17" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 42229 } } peers { permanent_uuid: "ee3ec60b51b44ca995c23c3908f540b5" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38893 } }
I20260805 23:10:18.102986 28242 tablet_bootstrap.cc:492] T 0a5661662ac2408ba029dfb031dac83c P ee3ec60b51b44ca995c23c3908f540b5: Bootstrap starting.
I20260805 23:10:18.103881 28243 ts_tablet_manager.cc:1428] T 0a5661662ac2408ba029dfb031dac83c P a2b8471244eb4bb4825f255d21e3cdba: Time spent starting tablet: real 0.007s user 0.001s sys 0.000s
I20260805 23:10:18.104200 28242 tablet_bootstrap.cc:654] T 0a5661662ac2408ba029dfb031dac83c P ee3ec60b51b44ca995c23c3908f540b5: Neither blocks nor log segments found. Creating new log.
I20260805 23:10:18.106434 28244 tablet_bootstrap.cc:492] T 0a5661662ac2408ba029dfb031dac83c P 3fa2896d2f224c69b1bb196eb4619a17: No bootstrap required, opened a new log
I20260805 23:10:18.106559 28244 ts_tablet_manager.cc:1397] T 0a5661662ac2408ba029dfb031dac83c P 3fa2896d2f224c69b1bb196eb4619a17: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260805 23:10:18.106608 28292 tablet_replica.cc:330] T 91f3ece73a5a47839322fdabdc40cc52 P 3fa2896d2f224c69b1bb196eb4619a17: stopping tablet replica
I20260805 23:10:18.106864 28242 tablet_bootstrap.cc:492] T 0a5661662ac2408ba029dfb031dac83c P ee3ec60b51b44ca995c23c3908f540b5: No bootstrap required, opened a new log
I20260805 23:10:18.106945 28242 ts_tablet_manager.cc:1397] T 0a5661662ac2408ba029dfb031dac83c P ee3ec60b51b44ca995c23c3908f540b5: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260805 23:10:18.106966 28292 raft_consensus.cc:2229] T 91f3ece73a5a47839322fdabdc40cc52 P 3fa2896d2f224c69b1bb196eb4619a17 [term 1 LEADER]: Raft consensus shutting down.
I20260805 23:10:18.107064 28244 raft_consensus.cc:348] T 0a5661662ac2408ba029dfb031dac83c P 3fa2896d2f224c69b1bb196eb4619a17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2b8471244eb4bb4825f255d21e3cdba" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 34855 } } peers { permanent_uuid: "3fa2896d2f224c69b1bb196eb4619a17" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 42229 } } peers { permanent_uuid: "ee3ec60b51b44ca995c23c3908f540b5" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38893 } }
I20260805 23:10:18.107177 28244 raft_consensus.cc:374] T 0a5661662ac2408ba029dfb031dac83c P 3fa2896d2f224c69b1bb196eb4619a17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:10:18.107209 28244 raft_consensus.cc:729] T 0a5661662ac2408ba029dfb031dac83c P 3fa2896d2f224c69b1bb196eb4619a17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fa2896d2f224c69b1bb196eb4619a17, State: Initialized, Role: FOLLOWER
I20260805 23:10:18.107388 28292 raft_consensus.cc:2258] T 91f3ece73a5a47839322fdabdc40cc52 P 3fa2896d2f224c69b1bb196eb4619a17 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:10:18.107368 28244 consensus_queue.cc:260] T 0a5661662ac2408ba029dfb031dac83c P 3fa2896d2f224c69b1bb196eb4619a17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2b8471244eb4bb4825f255d21e3cdba" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 34855 } } peers { permanent_uuid: "3fa2896d2f224c69b1bb196eb4619a17" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 42229 } } peers { permanent_uuid: "ee3ec60b51b44ca995c23c3908f540b5" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38893 } }
I20260805 23:10:18.107380 28242 raft_consensus.cc:348] T 0a5661662ac2408ba029dfb031dac83c P ee3ec60b51b44ca995c23c3908f540b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2b8471244eb4bb4825f255d21e3cdba" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 34855 } } peers { permanent_uuid: "3fa2896d2f224c69b1bb196eb4619a17" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 42229 } } peers { permanent_uuid: "ee3ec60b51b44ca995c23c3908f540b5" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38893 } }
I20260805 23:10:18.107506 28242 raft_consensus.cc:374] T 0a5661662ac2408ba029dfb031dac83c P ee3ec60b51b44ca995c23c3908f540b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:10:18.107537 28242 raft_consensus.cc:729] T 0a5661662ac2408ba029dfb031dac83c P ee3ec60b51b44ca995c23c3908f540b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee3ec60b51b44ca995c23c3908f540b5, State: Initialized, Role: FOLLOWER
I20260805 23:10:18.107683 28244 ts_tablet_manager.cc:1428] T 0a5661662ac2408ba029dfb031dac83c P 3fa2896d2f224c69b1bb196eb4619a17: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260805 23:10:18.107681 28242 consensus_queue.cc:260] T 0a5661662ac2408ba029dfb031dac83c P ee3ec60b51b44ca995c23c3908f540b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2b8471244eb4bb4825f255d21e3cdba" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 34855 } } peers { permanent_uuid: "3fa2896d2f224c69b1bb196eb4619a17" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 42229 } } peers { permanent_uuid: "ee3ec60b51b44ca995c23c3908f540b5" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38893 } }
I20260805 23:10:18.107761 28293 tablet_replica.cc:330] T 91f3ece73a5a47839322fdabdc40cc52 P a2b8471244eb4bb4825f255d21e3cdba: stopping tablet replica
I20260805 23:10:18.107961 28242 ts_tablet_manager.cc:1428] T 0a5661662ac2408ba029dfb031dac83c P ee3ec60b51b44ca995c23c3908f540b5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260805 23:10:18.108086 28293 raft_consensus.cc:2229] T 91f3ece73a5a47839322fdabdc40cc52 P a2b8471244eb4bb4825f255d21e3cdba [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 23:10:18.108273 28293 raft_consensus.cc:2258] T 91f3ece73a5a47839322fdabdc40cc52 P a2b8471244eb4bb4825f255d21e3cdba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:10:18.112262 28292 ts_tablet_manager.cc:1905] T 91f3ece73a5a47839322fdabdc40cc52 P 3fa2896d2f224c69b1bb196eb4619a17: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:10:18.114042 28293 ts_tablet_manager.cc:1905] T 91f3ece73a5a47839322fdabdc40cc52 P a2b8471244eb4bb4825f255d21e3cdba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:10:18.118130 28292 ts_tablet_manager.cc:1918] T 91f3ece73a5a47839322fdabdc40cc52 P 3fa2896d2f224c69b1bb196eb4619a17: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260805 23:10:18.118253 28292 log.cc:1196] T 91f3ece73a5a47839322fdabdc40cc52 P 3fa2896d2f224c69b1bb196eb4619a17: Deleting WAL directory at /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/minicluster-data/ts-1/wal/wals/91f3ece73a5a47839322fdabdc40cc52
I20260805 23:10:18.121239 28293 ts_tablet_manager.cc:1918] T 91f3ece73a5a47839322fdabdc40cc52 P a2b8471244eb4bb4825f255d21e3cdba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260805 23:10:18.121349 28293 log.cc:1196] T 91f3ece73a5a47839322fdabdc40cc52 P a2b8471244eb4bb4825f255d21e3cdba: Deleting WAL directory at /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/minicluster-data/ts-0/wal/wals/91f3ece73a5a47839322fdabdc40cc52
I20260805 23:10:18.121701 28293 ts_tablet_manager.cc:1939] T 91f3ece73a5a47839322fdabdc40cc52 P a2b8471244eb4bb4825f255d21e3cdba: Deleting consensus metadata
I20260805 23:10:18.123013 27712 catalog_manager.cc:4841] TS a2b8471244eb4bb4825f255d21e3cdba (127.25.92.193:34855): tablet 91f3ece73a5a47839322fdabdc40cc52 (table test-table [id=69d9b3bdd65648418911b2ec3e39bd84]) successfully deleted
I20260805 23:10:18.123220 28292 ts_tablet_manager.cc:1939] T 91f3ece73a5a47839322fdabdc40cc52 P 3fa2896d2f224c69b1bb196eb4619a17: Deleting consensus metadata
I20260805 23:10:18.124416 27711 catalog_manager.cc:4841] TS 3fa2896d2f224c69b1bb196eb4619a17 (127.25.92.194:42229): tablet 91f3ece73a5a47839322fdabdc40cc52 (table test-table [id=69d9b3bdd65648418911b2ec3e39bd84]) successfully deleted
I20260805 23:10:18.272183 28288 raft_consensus.cc:482] T 0a5661662ac2408ba029dfb031dac83c P 3fa2896d2f224c69b1bb196eb4619a17 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 23:10:18.272339 28288 raft_consensus.cc:504] T 0a5661662ac2408ba029dfb031dac83c P 3fa2896d2f224c69b1bb196eb4619a17 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2b8471244eb4bb4825f255d21e3cdba" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 34855 } } peers { permanent_uuid: "3fa2896d2f224c69b1bb196eb4619a17" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 42229 } } peers { permanent_uuid: "ee3ec60b51b44ca995c23c3908f540b5" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38893 } }
I20260805 23:10:18.272867 28288 leader_election.cc:288] T 0a5661662ac2408ba029dfb031dac83c P 3fa2896d2f224c69b1bb196eb4619a17 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2b8471244eb4bb4825f255d21e3cdba (127.25.92.193:34855), ee3ec60b51b44ca995c23c3908f540b5 (127.25.92.195:38893)
I20260805 23:10:18.273370 28172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a5661662ac2408ba029dfb031dac83c" candidate_uuid: "3fa2896d2f224c69b1bb196eb4619a17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee3ec60b51b44ca995c23c3908f540b5" is_pre_election: true
I20260805 23:10:18.273373 27888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a5661662ac2408ba029dfb031dac83c" candidate_uuid: "3fa2896d2f224c69b1bb196eb4619a17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b8471244eb4bb4825f255d21e3cdba" is_pre_election: true
I20260805 23:10:18.273553 28172 raft_consensus.cc:2454] T 0a5661662ac2408ba029dfb031dac83c P ee3ec60b51b44ca995c23c3908f540b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3fa2896d2f224c69b1bb196eb4619a17 in term 0.
I20260805 23:10:18.273587 27888 raft_consensus.cc:2454] T 0a5661662ac2408ba029dfb031dac83c P a2b8471244eb4bb4825f255d21e3cdba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3fa2896d2f224c69b1bb196eb4619a17 in term 0.
I20260805 23:10:18.273909 27961 leader_election.cc:302] T 0a5661662ac2408ba029dfb031dac83c P 3fa2896d2f224c69b1bb196eb4619a17 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fa2896d2f224c69b1bb196eb4619a17, ee3ec60b51b44ca995c23c3908f540b5; no voters:
I20260805 23:10:18.274158 28288 raft_consensus.cc:2781] T 0a5661662ac2408ba029dfb031dac83c P 3fa2896d2f224c69b1bb196eb4619a17 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260805 23:10:18.274236 28288 raft_consensus.cc:482] T 0a5661662ac2408ba029dfb031dac83c P 3fa2896d2f224c69b1bb196eb4619a17 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260805 23:10:18.274277 28288 raft_consensus.cc:3037] T 0a5661662ac2408ba029dfb031dac83c P 3fa2896d2f224c69b1bb196eb4619a17 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:10:18.275342 28288 raft_consensus.cc:504] T 0a5661662ac2408ba029dfb031dac83c P 3fa2896d2f224c69b1bb196eb4619a17 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2b8471244eb4bb4825f255d21e3cdba" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 34855 } } peers { permanent_uuid: "3fa2896d2f224c69b1bb196eb4619a17" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 42229 } } peers { permanent_uuid: "ee3ec60b51b44ca995c23c3908f540b5" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38893 } }
I20260805 23:10:18.275759 28288 leader_election.cc:288] T 0a5661662ac2408ba029dfb031dac83c P 3fa2896d2f224c69b1bb196eb4619a17 [CANDIDATE]: Term 1 election: Requested vote from peers a2b8471244eb4bb4825f255d21e3cdba (127.25.92.193:34855), ee3ec60b51b44ca995c23c3908f540b5 (127.25.92.195:38893)
I20260805 23:10:18.276135 27888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a5661662ac2408ba029dfb031dac83c" candidate_uuid: "3fa2896d2f224c69b1bb196eb4619a17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b8471244eb4bb4825f255d21e3cdba"
I20260805 23:10:18.276221 28172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a5661662ac2408ba029dfb031dac83c" candidate_uuid: "3fa2896d2f224c69b1bb196eb4619a17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee3ec60b51b44ca995c23c3908f540b5"
I20260805 23:10:18.276274 27888 raft_consensus.cc:3037] T 0a5661662ac2408ba029dfb031dac83c P a2b8471244eb4bb4825f255d21e3cdba [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:10:18.276358 28172 raft_consensus.cc:3037] T 0a5661662ac2408ba029dfb031dac83c P ee3ec60b51b44ca995c23c3908f540b5 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:10:18.277446 28172 raft_consensus.cc:2454] T 0a5661662ac2408ba029dfb031dac83c P ee3ec60b51b44ca995c23c3908f540b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fa2896d2f224c69b1bb196eb4619a17 in term 1.
I20260805 23:10:18.277446 27888 raft_consensus.cc:2454] T 0a5661662ac2408ba029dfb031dac83c P a2b8471244eb4bb4825f255d21e3cdba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fa2896d2f224c69b1bb196eb4619a17 in term 1.
I20260805 23:10:18.277820 27961 leader_election.cc:302] T 0a5661662ac2408ba029dfb031dac83c P 3fa2896d2f224c69b1bb196eb4619a17 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fa2896d2f224c69b1bb196eb4619a17, ee3ec60b51b44ca995c23c3908f540b5; no voters:
I20260805 23:10:18.278016 28288 raft_consensus.cc:2781] T 0a5661662ac2408ba029dfb031dac83c P 3fa2896d2f224c69b1bb196eb4619a17 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:10:18.278101 28288 raft_consensus.cc:686] T 0a5661662ac2408ba029dfb031dac83c P 3fa2896d2f224c69b1bb196eb4619a17 [term 1 LEADER]: Becoming Leader. State: Replica: 3fa2896d2f224c69b1bb196eb4619a17, State: Running, Role: LEADER
I20260805 23:10:18.278263 28288 consensus_queue.cc:237] T 0a5661662ac2408ba029dfb031dac83c P 3fa2896d2f224c69b1bb196eb4619a17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2b8471244eb4bb4825f255d21e3cdba" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 34855 } } peers { permanent_uuid: "3fa2896d2f224c69b1bb196eb4619a17" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 42229 } } peers { permanent_uuid: "ee3ec60b51b44ca995c23c3908f540b5" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38893 } }
I20260805 23:10:18.279878 27728 catalog_manager.cc:5495] T 0a5661662ac2408ba029dfb031dac83c P 3fa2896d2f224c69b1bb196eb4619a17 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3fa2896d2f224c69b1bb196eb4619a17 (127.25.92.194). New cstate: current_term: 1 leader_uuid: "3fa2896d2f224c69b1bb196eb4619a17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2b8471244eb4bb4825f255d21e3cdba" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3fa2896d2f224c69b1bb196eb4619a17" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 42229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee3ec60b51b44ca995c23c3908f540b5" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38893 } health_report { overall_health: UNKNOWN } } }
I20260805 23:10:18.309542 28172 raft_consensus.cc:1261] T 0a5661662ac2408ba029dfb031dac83c P ee3ec60b51b44ca995c23c3908f540b5 [term 1 FOLLOWER]: Refusing update from remote peer 3fa2896d2f224c69b1bb196eb4619a17: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:10:18.309747 27888 raft_consensus.cc:1261] T 0a5661662ac2408ba029dfb031dac83c P a2b8471244eb4bb4825f255d21e3cdba [term 1 FOLLOWER]: Refusing update from remote peer 3fa2896d2f224c69b1bb196eb4619a17: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:10:18.309955 28288 consensus_queue.cc:1035] T 0a5661662ac2408ba029dfb031dac83c P 3fa2896d2f224c69b1bb196eb4619a17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee3ec60b51b44ca995c23c3908f540b5" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38893 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:10:18.310174 28252 consensus_queue.cc:1035] T 0a5661662ac2408ba029dfb031dac83c P 3fa2896d2f224c69b1bb196eb4619a17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b8471244eb4bb4825f255d21e3cdba" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:10:18.313149 28271 mvcc.cc:205] Tried to move back new op lower bound from 7315338929392234496 to 7315338929268998144. Current Snapshot: MvccSnapshot[applied={T|T < 7315338929392234496}]
W20260805 23:10:18.325517 28272 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260805 23:10:19.884620 27728 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52132:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:10:19.884790 27728 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52132:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:10:19.887619 27728 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ea219c710c774b9d9e270e6fad216a05: Sending DeleteTablet for 3 replicas of tablet 0a5661662ac2408ba029dfb031dac83c
I20260805 23:10:19.888329 28010 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a5661662ac2408ba029dfb031dac83c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:10:19 UTC) from {username='kudu', principal='kudu/127.25.92.254@KRBTEST.COM'} at 127.0.0.1:33210
I20260805 23:10:19.888329 27868 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a5661662ac2408ba029dfb031dac83c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:10:19 UTC) from {username='kudu', principal='kudu/127.25.92.254@KRBTEST.COM'} at 127.0.0.1:49550
I20260805 23:10:19.888731 28152 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a5661662ac2408ba029dfb031dac83c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:10:19 UTC) from {username='kudu', principal='kudu/127.25.92.254@KRBTEST.COM'} at 127.0.0.1:46668
I20260805 23:10:19.889391 28322 tablet_replica.cc:330] T 0a5661662ac2408ba029dfb031dac83c P 3fa2896d2f224c69b1bb196eb4619a17: stopping tablet replica
I20260805 23:10:19.889621 28322 raft_consensus.cc:2229] T 0a5661662ac2408ba029dfb031dac83c P 3fa2896d2f224c69b1bb196eb4619a17 [term 1 LEADER]: Raft consensus shutting down.
I20260805 23:10:19.889875 28322 raft_consensus.cc:2258] T 0a5661662ac2408ba029dfb031dac83c P 3fa2896d2f224c69b1bb196eb4619a17 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:10:19.890174 28323 tablet_replica.cc:330] T 0a5661662ac2408ba029dfb031dac83c P a2b8471244eb4bb4825f255d21e3cdba: stopping tablet replica
I20260805 23:10:19.890345 28323 raft_consensus.cc:2229] T 0a5661662ac2408ba029dfb031dac83c P a2b8471244eb4bb4825f255d21e3cdba [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 23:10:19.890496 28323 raft_consensus.cc:2258] T 0a5661662ac2408ba029dfb031dac83c P a2b8471244eb4bb4825f255d21e3cdba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:10:19.891049 28324 tablet_replica.cc:330] T 0a5661662ac2408ba029dfb031dac83c P ee3ec60b51b44ca995c23c3908f540b5: stopping tablet replica
I20260805 23:10:19.891150 28322 ts_tablet_manager.cc:1905] T 0a5661662ac2408ba029dfb031dac83c P 3fa2896d2f224c69b1bb196eb4619a17: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:10:19.891197 28324 raft_consensus.cc:2229] T 0a5661662ac2408ba029dfb031dac83c P ee3ec60b51b44ca995c23c3908f540b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 23:10:19.891330 28324 raft_consensus.cc:2258] T 0a5661662ac2408ba029dfb031dac83c P ee3ec60b51b44ca995c23c3908f540b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:10:19.892041 28324 ts_tablet_manager.cc:1905] T 0a5661662ac2408ba029dfb031dac83c P ee3ec60b51b44ca995c23c3908f540b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:10:19.892086 28323 ts_tablet_manager.cc:1905] T 0a5661662ac2408ba029dfb031dac83c P a2b8471244eb4bb4825f255d21e3cdba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:10:19.894344 28322 ts_tablet_manager.cc:1918] T 0a5661662ac2408ba029dfb031dac83c P 3fa2896d2f224c69b1bb196eb4619a17: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260805 23:10:19.894440 28322 log.cc:1196] T 0a5661662ac2408ba029dfb031dac83c P 3fa2896d2f224c69b1bb196eb4619a17: Deleting WAL directory at /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/minicluster-data/ts-1/wal/wals/0a5661662ac2408ba029dfb031dac83c
I20260805 23:10:19.894771 28322 ts_tablet_manager.cc:1939] T 0a5661662ac2408ba029dfb031dac83c P 3fa2896d2f224c69b1bb196eb4619a17: Deleting consensus metadata
I20260805 23:10:19.895390 28323 ts_tablet_manager.cc:1918] T 0a5661662ac2408ba029dfb031dac83c P a2b8471244eb4bb4825f255d21e3cdba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260805 23:10:19.895473 28323 log.cc:1196] T 0a5661662ac2408ba029dfb031dac83c P a2b8471244eb4bb4825f255d21e3cdba: Deleting WAL directory at /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/minicluster-data/ts-0/wal/wals/0a5661662ac2408ba029dfb031dac83c
I20260805 23:10:19.895622 28324 ts_tablet_manager.cc:1918] T 0a5661662ac2408ba029dfb031dac83c P ee3ec60b51b44ca995c23c3908f540b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260805 23:10:19.895705 28324 log.cc:1196] T 0a5661662ac2408ba029dfb031dac83c P ee3ec60b51b44ca995c23c3908f540b5: Deleting WAL directory at /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/minicluster-data/ts-2/wal/wals/0a5661662ac2408ba029dfb031dac83c
I20260805 23:10:19.895788 28323 ts_tablet_manager.cc:1939] T 0a5661662ac2408ba029dfb031dac83c P a2b8471244eb4bb4825f255d21e3cdba: Deleting consensus metadata
I20260805 23:10:19.895820 27711 catalog_manager.cc:4841] TS 3fa2896d2f224c69b1bb196eb4619a17 (127.25.92.194:42229): tablet 0a5661662ac2408ba029dfb031dac83c (table test-table [id=0578120a5b864013a5243eeeaeaed5ec]) successfully deleted
I20260805 23:10:19.895982 28324 ts_tablet_manager.cc:1939] T 0a5661662ac2408ba029dfb031dac83c P ee3ec60b51b44ca995c23c3908f540b5: Deleting consensus metadata
I20260805 23:10:19.896577 27712 catalog_manager.cc:4841] TS a2b8471244eb4bb4825f255d21e3cdba (127.25.92.193:34855): tablet 0a5661662ac2408ba029dfb031dac83c (table test-table [id=0578120a5b864013a5243eeeaeaed5ec]) successfully deleted
I20260805 23:10:19.896799 27711 catalog_manager.cc:4841] TS ee3ec60b51b44ca995c23c3908f540b5 (127.25.92.195:38893): tablet 0a5661662ac2408ba029dfb031dac83c (table test-table [id=0578120a5b864013a5243eeeaeaed5ec]) successfully deleted
Aug 05 23:10:19 dist-test-slave-ss4n krb5kdc[27682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971419, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 05 23:10:19 dist-test-slave-ss4n krb5kdc[27682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971419, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM
I20260805 23:10:19.926944 28332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:19.909809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52142 (local address 127.25.92.254:38325)
0805 23:10:19.910154 (+ 345us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:19.910169 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:19.910281 (+ 112us) server_negotiation.cc:407] Connection header received
0805 23:10:19.910507 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:19.910511 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:19.910579 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:19.910698 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:19.911488 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:19.912120 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:19.913087 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:19.913355 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:19.913563 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:19.922337 (+ 8774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:19.922341 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:19.922343 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:19.922403 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:19.924313 (+ 1910us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:19.924864 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:19.924872 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:19.924874 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:19.924979 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:19.925318 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:19.925322 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:19.925324 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:19.925648 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:19.925794 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:19.926597 (+ 803us) server_negotiation.cc:299] Negotiation successful
0805 23:10:19.926748 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":121,"threads_started":1}
I20260805 23:10:19.931787 27728 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52142:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260805 23:10:19.932338 27728 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260805 23:10:19.938367 27868 tablet_service.cc:1467] Processing CreateTablet for tablet 72d8d9b583ff431e9e8b180073eecff0 (DEFAULT_TABLE table=test-table [id=5cc6f972031d4effa147e58f26b34605]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:10:19.938390 28010 tablet_service.cc:1467] Processing CreateTablet for tablet 72d8d9b583ff431e9e8b180073eecff0 (DEFAULT_TABLE table=test-table [id=5cc6f972031d4effa147e58f26b34605]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:10:19.938730 27868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72d8d9b583ff431e9e8b180073eecff0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:10:19.938759 28010 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72d8d9b583ff431e9e8b180073eecff0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:10:19.938731 28152 tablet_service.cc:1467] Processing CreateTablet for tablet 72d8d9b583ff431e9e8b180073eecff0 (DEFAULT_TABLE table=test-table [id=5cc6f972031d4effa147e58f26b34605]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:10:19.939045 28152 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72d8d9b583ff431e9e8b180073eecff0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:10:19.941782 28335 tablet_bootstrap.cc:492] T 72d8d9b583ff431e9e8b180073eecff0 P ee3ec60b51b44ca995c23c3908f540b5: Bootstrap starting.
I20260805 23:10:19.941829 28333 tablet_bootstrap.cc:492] T 72d8d9b583ff431e9e8b180073eecff0 P a2b8471244eb4bb4825f255d21e3cdba: Bootstrap starting.
I20260805 23:10:19.942754 28335 tablet_bootstrap.cc:654] T 72d8d9b583ff431e9e8b180073eecff0 P ee3ec60b51b44ca995c23c3908f540b5: Neither blocks nor log segments found. Creating new log.
I20260805 23:10:19.942757 28333 tablet_bootstrap.cc:654] T 72d8d9b583ff431e9e8b180073eecff0 P a2b8471244eb4bb4825f255d21e3cdba: Neither blocks nor log segments found. Creating new log.
I20260805 23:10:19.943447 28334 tablet_bootstrap.cc:492] T 72d8d9b583ff431e9e8b180073eecff0 P 3fa2896d2f224c69b1bb196eb4619a17: Bootstrap starting.
I20260805 23:10:19.944290 28335 tablet_bootstrap.cc:492] T 72d8d9b583ff431e9e8b180073eecff0 P ee3ec60b51b44ca995c23c3908f540b5: No bootstrap required, opened a new log
I20260805 23:10:19.944456 28335 ts_tablet_manager.cc:1397] T 72d8d9b583ff431e9e8b180073eecff0 P ee3ec60b51b44ca995c23c3908f540b5: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260805 23:10:19.944499 28334 tablet_bootstrap.cc:654] T 72d8d9b583ff431e9e8b180073eecff0 P 3fa2896d2f224c69b1bb196eb4619a17: Neither blocks nor log segments found. Creating new log.
I20260805 23:10:19.945027 28335 raft_consensus.cc:348] T 72d8d9b583ff431e9e8b180073eecff0 P ee3ec60b51b44ca995c23c3908f540b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2b8471244eb4bb4825f255d21e3cdba" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 34855 } } peers { permanent_uuid: "3fa2896d2f224c69b1bb196eb4619a17" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 42229 } } peers { permanent_uuid: "ee3ec60b51b44ca995c23c3908f540b5" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38893 } }
I20260805 23:10:19.945185 28335 raft_consensus.cc:374] T 72d8d9b583ff431e9e8b180073eecff0 P ee3ec60b51b44ca995c23c3908f540b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:10:19.945230 28335 raft_consensus.cc:729] T 72d8d9b583ff431e9e8b180073eecff0 P ee3ec60b51b44ca995c23c3908f540b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee3ec60b51b44ca995c23c3908f540b5, State: Initialized, Role: FOLLOWER
I20260805 23:10:19.945381 28335 consensus_queue.cc:260] T 72d8d9b583ff431e9e8b180073eecff0 P ee3ec60b51b44ca995c23c3908f540b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2b8471244eb4bb4825f255d21e3cdba" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 34855 } } peers { permanent_uuid: "3fa2896d2f224c69b1bb196eb4619a17" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 42229 } } peers { permanent_uuid: "ee3ec60b51b44ca995c23c3908f540b5" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38893 } }
I20260805 23:10:19.945839 28333 tablet_bootstrap.cc:492] T 72d8d9b583ff431e9e8b180073eecff0 P a2b8471244eb4bb4825f255d21e3cdba: No bootstrap required, opened a new log
I20260805 23:10:19.945896 28335 ts_tablet_manager.cc:1428] T 72d8d9b583ff431e9e8b180073eecff0 P ee3ec60b51b44ca995c23c3908f540b5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260805 23:10:19.945948 28333 ts_tablet_manager.cc:1397] T 72d8d9b583ff431e9e8b180073eecff0 P a2b8471244eb4bb4825f255d21e3cdba: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260805 23:10:19.946346 28333 raft_consensus.cc:348] T 72d8d9b583ff431e9e8b180073eecff0 P a2b8471244eb4bb4825f255d21e3cdba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2b8471244eb4bb4825f255d21e3cdba" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 34855 } } peers { permanent_uuid: "3fa2896d2f224c69b1bb196eb4619a17" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 42229 } } peers { permanent_uuid: "ee3ec60b51b44ca995c23c3908f540b5" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38893 } }
I20260805 23:10:19.946448 28333 raft_consensus.cc:374] T 72d8d9b583ff431e9e8b180073eecff0 P a2b8471244eb4bb4825f255d21e3cdba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:10:19.946502 28333 raft_consensus.cc:729] T 72d8d9b583ff431e9e8b180073eecff0 P a2b8471244eb4bb4825f255d21e3cdba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2b8471244eb4bb4825f255d21e3cdba, State: Initialized, Role: FOLLOWER
I20260805 23:10:19.946650 28333 consensus_queue.cc:260] T 72d8d9b583ff431e9e8b180073eecff0 P a2b8471244eb4bb4825f255d21e3cdba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2b8471244eb4bb4825f255d21e3cdba" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 34855 } } peers { permanent_uuid: "3fa2896d2f224c69b1bb196eb4619a17" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 42229 } } peers { permanent_uuid: "ee3ec60b51b44ca995c23c3908f540b5" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38893 } }
I20260805 23:10:19.947405 28333 ts_tablet_manager.cc:1428] T 72d8d9b583ff431e9e8b180073eecff0 P a2b8471244eb4bb4825f255d21e3cdba: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260805 23:10:19.948026 28334 tablet_bootstrap.cc:492] T 72d8d9b583ff431e9e8b180073eecff0 P 3fa2896d2f224c69b1bb196eb4619a17: No bootstrap required, opened a new log
I20260805 23:10:19.948164 28334 ts_tablet_manager.cc:1397] T 72d8d9b583ff431e9e8b180073eecff0 P 3fa2896d2f224c69b1bb196eb4619a17: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260805 23:10:19.949123 28334 raft_consensus.cc:348] T 72d8d9b583ff431e9e8b180073eecff0 P 3fa2896d2f224c69b1bb196eb4619a17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2b8471244eb4bb4825f255d21e3cdba" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 34855 } } peers { permanent_uuid: "3fa2896d2f224c69b1bb196eb4619a17" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 42229 } } peers { permanent_uuid: "ee3ec60b51b44ca995c23c3908f540b5" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38893 } }
I20260805 23:10:19.949230 28334 raft_consensus.cc:374] T 72d8d9b583ff431e9e8b180073eecff0 P 3fa2896d2f224c69b1bb196eb4619a17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:10:19.949291 28334 raft_consensus.cc:729] T 72d8d9b583ff431e9e8b180073eecff0 P 3fa2896d2f224c69b1bb196eb4619a17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fa2896d2f224c69b1bb196eb4619a17, State: Initialized, Role: FOLLOWER
I20260805 23:10:19.949498 28334 consensus_queue.cc:260] T 72d8d9b583ff431e9e8b180073eecff0 P 3fa2896d2f224c69b1bb196eb4619a17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2b8471244eb4bb4825f255d21e3cdba" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 34855 } } peers { permanent_uuid: "3fa2896d2f224c69b1bb196eb4619a17" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 42229 } } peers { permanent_uuid: "ee3ec60b51b44ca995c23c3908f540b5" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38893 } }
I20260805 23:10:19.949831 28334 ts_tablet_manager.cc:1428] T 72d8d9b583ff431e9e8b180073eecff0 P 3fa2896d2f224c69b1bb196eb4619a17: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260805 23:10:19.961985 28339 raft_consensus.cc:482] T 72d8d9b583ff431e9e8b180073eecff0 P ee3ec60b51b44ca995c23c3908f540b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 23:10:19.962211 28339 raft_consensus.cc:504] T 72d8d9b583ff431e9e8b180073eecff0 P ee3ec60b51b44ca995c23c3908f540b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2b8471244eb4bb4825f255d21e3cdba" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 34855 } } peers { permanent_uuid: "3fa2896d2f224c69b1bb196eb4619a17" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 42229 } } peers { permanent_uuid: "ee3ec60b51b44ca995c23c3908f540b5" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38893 } }
I20260805 23:10:19.963675 28339 leader_election.cc:288] T 72d8d9b583ff431e9e8b180073eecff0 P ee3ec60b51b44ca995c23c3908f540b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2b8471244eb4bb4825f255d21e3cdba (127.25.92.193:34855), 3fa2896d2f224c69b1bb196eb4619a17 (127.25.92.194:42229)
I20260805 23:10:19.968340 28343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:19.964160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.195:58845 (local address 127.25.92.194:42229)
0805 23:10:19.964508 (+ 348us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:19.964519 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:19.964540 (+ 21us) server_negotiation.cc:407] Connection header received
0805 23:10:19.964735 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:19.964738 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:19.964819 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:19.964941 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:10:19.965198 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:19.966032 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:19.967198 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:19.967665 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:19.967799 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:19.967855 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:19.968125 (+ 270us) server_negotiation.cc:299] Negotiation successful
0805 23:10:19.968216 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":268,"thread_start_us":127,"threads_started":1}
I20260805 23:10:19.968305 28341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:19.964032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.194:42229 (local address 127.25.92.195:58845)
0805 23:10:19.964456 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:19.964470 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:19.964570 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:19.964942 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:19.964944 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:19.964959 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:10:19.965063 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:19.965068 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:19.966176 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:19.966179 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:19.967050 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:19.967068 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:19.967792 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:19.967795 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:19.967931 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:19.967950 (+ 19us) client_negotiation.cc:769] Sending connection context
0805 23:10:19.968039 (+ 89us) client_negotiation.cc:240] Negotiation successful
0805 23:10:19.968129 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":354,"mutex_wait_us":1,"thread_start_us":131,"threads_started":1}
I20260805 23:10:19.968613 28340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:19.963874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.193:34855 (local address 127.25.92.195:60785)
0805 23:10:19.964326 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:19.964346 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:19.964516 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:19.964841 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:19.964847 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:19.964874 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:10:19.965046 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:19.965055 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:19.965973 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:19.965978 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:19.966983 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:19.966994 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:19.967794 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:19.967799 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:19.968002 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:19.968024 (+ 22us) client_negotiation.cc:769] Sending connection context
0805 23:10:19.968136 (+ 112us) client_negotiation.cc:240] Negotiation successful
0805 23:10:19.968225 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":346,"mutex_wait_us":62,"thread_start_us":148,"threads_started":1}
I20260805 23:10:19.968852 28030 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72d8d9b583ff431e9e8b180073eecff0" candidate_uuid: "ee3ec60b51b44ca995c23c3908f540b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fa2896d2f224c69b1bb196eb4619a17" is_pre_election: true
I20260805 23:10:19.968987 28342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:19.964035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.195:60785 (local address 127.25.92.193:34855)
0805 23:10:19.964371 (+ 336us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:19.964380 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:19.964460 (+ 80us) server_negotiation.cc:407] Connection header received
0805 23:10:19.964660 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:19.964663 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:19.964722 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:19.964789 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:10:19.965206 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:19.965848 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:19.967138 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:19.967641 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:19.967804 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:19.967878 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:19.968191 (+ 313us) server_negotiation.cc:299] Negotiation successful
0805 23:10:19.968280 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":250,"thread_start_us":131,"threads_started":1}
I20260805 23:10:19.969048 28030 raft_consensus.cc:2454] T 72d8d9b583ff431e9e8b180073eecff0 P 3fa2896d2f224c69b1bb196eb4619a17 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee3ec60b51b44ca995c23c3908f540b5 in term 0.
I20260805 23:10:19.969528 28102 leader_election.cc:302] T 72d8d9b583ff431e9e8b180073eecff0 P ee3ec60b51b44ca995c23c3908f540b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fa2896d2f224c69b1bb196eb4619a17, ee3ec60b51b44ca995c23c3908f540b5; no voters:
I20260805 23:10:19.969837 27888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72d8d9b583ff431e9e8b180073eecff0" candidate_uuid: "ee3ec60b51b44ca995c23c3908f540b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b8471244eb4bb4825f255d21e3cdba" is_pre_election: true
I20260805 23:10:19.969909 28339 raft_consensus.cc:2781] T 72d8d9b583ff431e9e8b180073eecff0 P ee3ec60b51b44ca995c23c3908f540b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260805 23:10:19.969988 28339 raft_consensus.cc:482] T 72d8d9b583ff431e9e8b180073eecff0 P ee3ec60b51b44ca995c23c3908f540b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260805 23:10:19.970026 27888 raft_consensus.cc:2454] T 72d8d9b583ff431e9e8b180073eecff0 P a2b8471244eb4bb4825f255d21e3cdba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee3ec60b51b44ca995c23c3908f540b5 in term 0.
I20260805 23:10:19.970049 28339 raft_consensus.cc:3037] T 72d8d9b583ff431e9e8b180073eecff0 P ee3ec60b51b44ca995c23c3908f540b5 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:10:19.971354 28339 raft_consensus.cc:504] T 72d8d9b583ff431e9e8b180073eecff0 P ee3ec60b51b44ca995c23c3908f540b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2b8471244eb4bb4825f255d21e3cdba" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 34855 } } peers { permanent_uuid: "3fa2896d2f224c69b1bb196eb4619a17" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 42229 } } peers { permanent_uuid: "ee3ec60b51b44ca995c23c3908f540b5" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38893 } }
I20260805 23:10:19.971855 28339 leader_election.cc:288] T 72d8d9b583ff431e9e8b180073eecff0 P ee3ec60b51b44ca995c23c3908f540b5 [CANDIDATE]: Term 1 election: Requested vote from peers a2b8471244eb4bb4825f255d21e3cdba (127.25.92.193:34855), 3fa2896d2f224c69b1bb196eb4619a17 (127.25.92.194:42229)
I20260805 23:10:19.972255 27888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72d8d9b583ff431e9e8b180073eecff0" candidate_uuid: "ee3ec60b51b44ca995c23c3908f540b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b8471244eb4bb4825f255d21e3cdba"
I20260805 23:10:19.972436 27888 raft_consensus.cc:3037] T 72d8d9b583ff431e9e8b180073eecff0 P a2b8471244eb4bb4825f255d21e3cdba [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:10:19.972380 28030 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72d8d9b583ff431e9e8b180073eecff0" candidate_uuid: "ee3ec60b51b44ca995c23c3908f540b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fa2896d2f224c69b1bb196eb4619a17"
I20260805 23:10:19.972664 28030 raft_consensus.cc:3037] T 72d8d9b583ff431e9e8b180073eecff0 P 3fa2896d2f224c69b1bb196eb4619a17 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:10:19.973675 27888 raft_consensus.cc:2454] T 72d8d9b583ff431e9e8b180073eecff0 P a2b8471244eb4bb4825f255d21e3cdba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee3ec60b51b44ca995c23c3908f540b5 in term 1.
I20260805 23:10:19.973858 28030 raft_consensus.cc:2454] T 72d8d9b583ff431e9e8b180073eecff0 P 3fa2896d2f224c69b1bb196eb4619a17 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee3ec60b51b44ca995c23c3908f540b5 in term 1.
I20260805 23:10:19.974069 28103 leader_election.cc:302] T 72d8d9b583ff431e9e8b180073eecff0 P ee3ec60b51b44ca995c23c3908f540b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2b8471244eb4bb4825f255d21e3cdba, ee3ec60b51b44ca995c23c3908f540b5; no voters:
I20260805 23:10:19.974319 28339 raft_consensus.cc:2781] T 72d8d9b583ff431e9e8b180073eecff0 P ee3ec60b51b44ca995c23c3908f540b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:10:19.974581 28339 raft_consensus.cc:686] T 72d8d9b583ff431e9e8b180073eecff0 P ee3ec60b51b44ca995c23c3908f540b5 [term 1 LEADER]: Becoming Leader. State: Replica: ee3ec60b51b44ca995c23c3908f540b5, State: Running, Role: LEADER
I20260805 23:10:19.974875 28339 consensus_queue.cc:237] T 72d8d9b583ff431e9e8b180073eecff0 P ee3ec60b51b44ca995c23c3908f540b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2b8471244eb4bb4825f255d21e3cdba" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 34855 } } peers { permanent_uuid: "3fa2896d2f224c69b1bb196eb4619a17" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 42229 } } peers { permanent_uuid: "ee3ec60b51b44ca995c23c3908f540b5" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38893 } }
I20260805 23:10:19.977351 27724 catalog_manager.cc:5495] T 72d8d9b583ff431e9e8b180073eecff0 P ee3ec60b51b44ca995c23c3908f540b5 reported cstate change: term changed from 0 to 1, leader changed from <none> to ee3ec60b51b44ca995c23c3908f540b5 (127.25.92.195). New cstate: current_term: 1 leader_uuid: "ee3ec60b51b44ca995c23c3908f540b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2b8471244eb4bb4825f255d21e3cdba" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3fa2896d2f224c69b1bb196eb4619a17" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 42229 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee3ec60b51b44ca995c23c3908f540b5" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38893 } health_report { overall_health: HEALTHY } } }
I20260805 23:10:20.004946 28347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:20.000296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46676 (local address 127.25.92.195:38893)
0805 23:10:20.000565 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:20.000572 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:20.000584 (+ 12us) server_negotiation.cc:407] Connection header received
0805 23:10:20.000667 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:20.000669 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:20.000762 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:20.000883 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 23:10:20.001426 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:20.002162 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:20.003156 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:20.003417 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:20.003516 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:20.003871 (+ 355us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 23:10:20.004375 (+ 504us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 23:10:20.004495 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:20.004716 (+ 221us) server_negotiation.cc:299] Negotiation successful
0805 23:10:20.004778 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":122,"threads_started":1}
I20260805 23:10:20.017155 28350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:20.012864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.193:34855 (local address 127.25.92.195:35281)
0805 23:10:20.013350 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:20.013364 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:20.013474 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:20.013825 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:20.013828 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:20.013835 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0805 23:10:20.013950 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:20.013955 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:20.014789 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:20.014792 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:20.015725 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:20.015738 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:20.016317 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:20.016320 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:20.016398 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:20.016446 (+ 48us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0805 23:10:20.016872 (+ 426us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0805 23:10:20.016879 (+ 7us) client_negotiation.cc:769] Sending connection context
0805 23:10:20.016941 (+ 62us) client_negotiation.cc:240] Negotiation successful
0805 23:10:20.017017 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":154,"threads_started":1}
I20260805 23:10:20.017149 28342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:20.013060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.195:35281 (local address 127.25.92.193:34855)
0805 23:10:20.013227 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:20.013231 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:20.013471 (+ 240us) server_negotiation.cc:407] Connection header received
0805 23:10:20.013673 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:20.013675 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:20.013724 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:20.013800 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 23:10:20.014045 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:20.014664 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:20.015870 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:20.016148 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:20.016253 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:20.016667 (+ 414us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 23:10:20.016760 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 23:10:20.016832 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:20.016970 (+ 138us) server_negotiation.cc:299] Negotiation successful
0805 23:10:20.017024 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260805 23:10:20.024286 27888 raft_consensus.cc:1261] T 72d8d9b583ff431e9e8b180073eecff0 P a2b8471244eb4bb4825f255d21e3cdba [term 1 FOLLOWER]: Refusing update from remote peer ee3ec60b51b44ca995c23c3908f540b5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:10:20.024781 28030 raft_consensus.cc:1261] T 72d8d9b583ff431e9e8b180073eecff0 P 3fa2896d2f224c69b1bb196eb4619a17 [term 1 FOLLOWER]: Refusing update from remote peer ee3ec60b51b44ca995c23c3908f540b5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:10:20.024843 28344 consensus_queue.cc:1035] T 72d8d9b583ff431e9e8b180073eecff0 P ee3ec60b51b44ca995c23c3908f540b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b8471244eb4bb4825f255d21e3cdba" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:10:20.025319 28344 consensus_queue.cc:1035] T 72d8d9b583ff431e9e8b180073eecff0 P ee3ec60b51b44ca995c23c3908f540b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fa2896d2f224c69b1bb196eb4619a17" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:10:20.047029 28354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:20.043082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.195:38893 (local address 127.25.92.193:50129)
0805 23:10:20.043382 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:20.043406 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:20.043505 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:20.043786 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:20.043789 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:20.043796 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0805 23:10:20.043897 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:20.043901 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:20.044810 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:20.044814 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:20.045684 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:20.045717 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:20.046224 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:20.046228 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:20.046358 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:20.046395 (+ 37us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0805 23:10:20.046705 (+ 310us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0805 23:10:20.046715 (+ 10us) client_negotiation.cc:769] Sending connection context
0805 23:10:20.046786 (+ 71us) client_negotiation.cc:240] Negotiation successful
0805 23:10:20.046884 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":134,"threads_started":1}
I20260805 23:10:20.047011 28347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:20.043227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.193:50129 (local address 127.25.92.195:38893)
0805 23:10:20.043344 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:20.043354 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:20.043460 (+ 106us) server_negotiation.cc:407] Connection header received
0805 23:10:20.043582 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:20.043585 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:20.043674 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:20.043763 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 23:10:20.043993 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:20.044673 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:20.045818 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:20.046084 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:20.046226 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:20.046518 (+ 292us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 23:10:20.046604 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 23:10:20.046697 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:20.046796 (+ 99us) server_negotiation.cc:299] Negotiation successful
0805 23:10:20.046856 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260805 23:10:20.074342 27724 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52142:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:10:20.074589 27724 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52142:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:10:20.077404 27724 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ea219c710c774b9d9e270e6fad216a05: Sending DeleteTablet for 3 replicas of tablet 72d8d9b583ff431e9e8b180073eecff0
I20260805 23:10:20.078168 27868 tablet_service.cc:1514] Processing DeleteTablet for tablet 72d8d9b583ff431e9e8b180073eecff0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:10:20 UTC) from {username='kudu', principal='kudu/127.25.92.254@KRBTEST.COM'} at 127.0.0.1:49550
I20260805 23:10:20.078348 28010 tablet_service.cc:1514] Processing DeleteTablet for tablet 72d8d9b583ff431e9e8b180073eecff0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:10:20 UTC) from {username='kudu', principal='kudu/127.25.92.254@KRBTEST.COM'} at 127.0.0.1:33210
I20260805 23:10:20.078447 28152 tablet_service.cc:1514] Processing DeleteTablet for tablet 72d8d9b583ff431e9e8b180073eecff0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:10:20 UTC) from {username='kudu', principal='kudu/127.25.92.254@KRBTEST.COM'} at 127.0.0.1:46668
I20260805 23:10:20.078653 28322 tablet_replica.cc:330] T 72d8d9b583ff431e9e8b180073eecff0 P 3fa2896d2f224c69b1bb196eb4619a17: stopping tablet replica
I20260805 23:10:20.078670 28324 tablet_replica.cc:330] T 72d8d9b583ff431e9e8b180073eecff0 P ee3ec60b51b44ca995c23c3908f540b5: stopping tablet replica
I20260805 23:10:20.078787 28322 raft_consensus.cc:2229] T 72d8d9b583ff431e9e8b180073eecff0 P 3fa2896d2f224c69b1bb196eb4619a17 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 23:10:20.078806 28324 raft_consensus.cc:2229] T 72d8d9b583ff431e9e8b180073eecff0 P ee3ec60b51b44ca995c23c3908f540b5 [term 1 LEADER]: Raft consensus shutting down.
I20260805 23:10:20.078938 28322 raft_consensus.cc:2258] T 72d8d9b583ff431e9e8b180073eecff0 P 3fa2896d2f224c69b1bb196eb4619a17 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:10:20.079190 28324 raft_consensus.cc:2258] T 72d8d9b583ff431e9e8b180073eecff0 P ee3ec60b51b44ca995c23c3908f540b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:10:20.079478 28323 tablet_replica.cc:330] T 72d8d9b583ff431e9e8b180073eecff0 P a2b8471244eb4bb4825f255d21e3cdba: stopping tablet replica
I20260805 23:10:20.079655 28323 raft_consensus.cc:2229] T 72d8d9b583ff431e9e8b180073eecff0 P a2b8471244eb4bb4825f255d21e3cdba [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 23:10:20.079807 28323 raft_consensus.cc:2258] T 72d8d9b583ff431e9e8b180073eecff0 P a2b8471244eb4bb4825f255d21e3cdba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:10:20.080087 28322 ts_tablet_manager.cc:1905] T 72d8d9b583ff431e9e8b180073eecff0 P 3fa2896d2f224c69b1bb196eb4619a17: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:10:20.080644 28323 ts_tablet_manager.cc:1905] T 72d8d9b583ff431e9e8b180073eecff0 P a2b8471244eb4bb4825f255d21e3cdba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:10:20.083271 28322 ts_tablet_manager.cc:1918] T 72d8d9b583ff431e9e8b180073eecff0 P 3fa2896d2f224c69b1bb196eb4619a17: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260805 23:10:20.083369 28322 log.cc:1196] T 72d8d9b583ff431e9e8b180073eecff0 P 3fa2896d2f224c69b1bb196eb4619a17: Deleting WAL directory at /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/minicluster-data/ts-1/wal/wals/72d8d9b583ff431e9e8b180073eecff0
I20260805 23:10:20.083473 28324 ts_tablet_manager.cc:1905] T 72d8d9b583ff431e9e8b180073eecff0 P ee3ec60b51b44ca995c23c3908f540b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:10:20.083684 28322 ts_tablet_manager.cc:1939] T 72d8d9b583ff431e9e8b180073eecff0 P 3fa2896d2f224c69b1bb196eb4619a17: Deleting consensus metadata
I20260805 23:10:20.083675 28323 ts_tablet_manager.cc:1918] T 72d8d9b583ff431e9e8b180073eecff0 P a2b8471244eb4bb4825f255d21e3cdba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260805 23:10:20.083772 28323 log.cc:1196] T 72d8d9b583ff431e9e8b180073eecff0 P a2b8471244eb4bb4825f255d21e3cdba: Deleting WAL directory at /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/minicluster-data/ts-0/wal/wals/72d8d9b583ff431e9e8b180073eecff0
I20260805 23:10:20.084071 28323 ts_tablet_manager.cc:1939] T 72d8d9b583ff431e9e8b180073eecff0 P a2b8471244eb4bb4825f255d21e3cdba: Deleting consensus metadata
I20260805 23:10:20.084584 27711 catalog_manager.cc:4841] TS 3fa2896d2f224c69b1bb196eb4619a17 (127.25.92.194:42229): tablet 72d8d9b583ff431e9e8b180073eecff0 (table test-table [id=5cc6f972031d4effa147e58f26b34605]) successfully deleted
I20260805 23:10:20.085080 27712 catalog_manager.cc:4841] TS a2b8471244eb4bb4825f255d21e3cdba (127.25.92.193:34855): tablet 72d8d9b583ff431e9e8b180073eecff0 (table test-table [id=5cc6f972031d4effa147e58f26b34605]) successfully deleted
I20260805 23:10:20.085412 27724 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52142:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260805 23:10:20.086032 27724 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260805 23:10:20.086787 28324 ts_tablet_manager.cc:1918] T 72d8d9b583ff431e9e8b180073eecff0 P ee3ec60b51b44ca995c23c3908f540b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260805 23:10:20.086910 28324 log.cc:1196] T 72d8d9b583ff431e9e8b180073eecff0 P ee3ec60b51b44ca995c23c3908f540b5: Deleting WAL directory at /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/minicluster-data/ts-2/wal/wals/72d8d9b583ff431e9e8b180073eecff0
I20260805 23:10:20.087215 28324 ts_tablet_manager.cc:1939] T 72d8d9b583ff431e9e8b180073eecff0 P ee3ec60b51b44ca995c23c3908f540b5: Deleting consensus metadata
I20260805 23:10:20.088433 27711 catalog_manager.cc:4841] TS ee3ec60b51b44ca995c23c3908f540b5 (127.25.92.195:38893): tablet 72d8d9b583ff431e9e8b180073eecff0 (table test-table [id=5cc6f972031d4effa147e58f26b34605]) successfully deleted
I20260805 23:10:20.092020 28152 tablet_service.cc:1467] Processing CreateTablet for tablet 906bbc2a7f06406da9a9094adfea8764 (DEFAULT_TABLE table=test-table [id=588d93a47eb142daa51320ded03c94d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:10:20.092005 28010 tablet_service.cc:1467] Processing CreateTablet for tablet 906bbc2a7f06406da9a9094adfea8764 (DEFAULT_TABLE table=test-table [id=588d93a47eb142daa51320ded03c94d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:10:20.092338 28152 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 906bbc2a7f06406da9a9094adfea8764. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:10:20.092365 28010 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 906bbc2a7f06406da9a9094adfea8764. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:10:20.093977 27868 tablet_service.cc:1467] Processing CreateTablet for tablet 906bbc2a7f06406da9a9094adfea8764 (DEFAULT_TABLE table=test-table [id=588d93a47eb142daa51320ded03c94d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:10:20.094278 27868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 906bbc2a7f06406da9a9094adfea8764. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:10:20.094951 28334 tablet_bootstrap.cc:492] T 906bbc2a7f06406da9a9094adfea8764 P 3fa2896d2f224c69b1bb196eb4619a17: Bootstrap starting.
I20260805 23:10:20.095891 28334 tablet_bootstrap.cc:654] T 906bbc2a7f06406da9a9094adfea8764 P 3fa2896d2f224c69b1bb196eb4619a17: Neither blocks nor log segments found. Creating new log.
I20260805 23:10:20.096509 28335 tablet_bootstrap.cc:492] T 906bbc2a7f06406da9a9094adfea8764 P ee3ec60b51b44ca995c23c3908f540b5: Bootstrap starting.
I20260805 23:10:20.097641 28333 tablet_bootstrap.cc:492] T 906bbc2a7f06406da9a9094adfea8764 P a2b8471244eb4bb4825f255d21e3cdba: Bootstrap starting.
I20260805 23:10:20.098598 28333 tablet_bootstrap.cc:654] T 906bbc2a7f06406da9a9094adfea8764 P a2b8471244eb4bb4825f255d21e3cdba: Neither blocks nor log segments found. Creating new log.
I20260805 23:10:20.097447 28335 tablet_bootstrap.cc:654] T 906bbc2a7f06406da9a9094adfea8764 P ee3ec60b51b44ca995c23c3908f540b5: Neither blocks nor log segments found. Creating new log.
I20260805 23:10:20.099865 28333 tablet_bootstrap.cc:492] T 906bbc2a7f06406da9a9094adfea8764 P a2b8471244eb4bb4825f255d21e3cdba: No bootstrap required, opened a new log
I20260805 23:10:20.099998 28333 ts_tablet_manager.cc:1397] T 906bbc2a7f06406da9a9094adfea8764 P a2b8471244eb4bb4825f255d21e3cdba: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260805 23:10:20.100426 28334 tablet_bootstrap.cc:492] T 906bbc2a7f06406da9a9094adfea8764 P 3fa2896d2f224c69b1bb196eb4619a17: No bootstrap required, opened a new log
I20260805 23:10:20.100541 28334 ts_tablet_manager.cc:1397] T 906bbc2a7f06406da9a9094adfea8764 P 3fa2896d2f224c69b1bb196eb4619a17: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260805 23:10:20.100616 28333 raft_consensus.cc:348] T 906bbc2a7f06406da9a9094adfea8764 P a2b8471244eb4bb4825f255d21e3cdba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2b8471244eb4bb4825f255d21e3cdba" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 34855 } } peers { permanent_uuid: "3fa2896d2f224c69b1bb196eb4619a17" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 42229 } } peers { permanent_uuid: "ee3ec60b51b44ca995c23c3908f540b5" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38893 } }
I20260805 23:10:20.100788 28333 raft_consensus.cc:374] T 906bbc2a7f06406da9a9094adfea8764 P a2b8471244eb4bb4825f255d21e3cdba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:10:20.100837 28333 raft_consensus.cc:729] T 906bbc2a7f06406da9a9094adfea8764 P a2b8471244eb4bb4825f255d21e3cdba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2b8471244eb4bb4825f255d21e3cdba, State: Initialized, Role: FOLLOWER
I20260805 23:10:20.101023 28334 raft_consensus.cc:348] T 906bbc2a7f06406da9a9094adfea8764 P 3fa2896d2f224c69b1bb196eb4619a17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2b8471244eb4bb4825f255d21e3cdba" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 34855 } } peers { permanent_uuid: "3fa2896d2f224c69b1bb196eb4619a17" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 42229 } } peers { permanent_uuid: "ee3ec60b51b44ca995c23c3908f540b5" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38893 } }
I20260805 23:10:20.101161 28334 raft_consensus.cc:374] T 906bbc2a7f06406da9a9094adfea8764 P 3fa2896d2f224c69b1bb196eb4619a17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:10:20.101197 28334 raft_consensus.cc:729] T 906bbc2a7f06406da9a9094adfea8764 P 3fa2896d2f224c69b1bb196eb4619a17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fa2896d2f224c69b1bb196eb4619a17, State: Initialized, Role: FOLLOWER
I20260805 23:10:20.100981 28333 consensus_queue.cc:260] T 906bbc2a7f06406da9a9094adfea8764 P a2b8471244eb4bb4825f255d21e3cdba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2b8471244eb4bb4825f255d21e3cdba" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 34855 } } peers { permanent_uuid: "3fa2896d2f224c69b1bb196eb4619a17" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 42229 } } peers { permanent_uuid: "ee3ec60b51b44ca995c23c3908f540b5" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38893 } }
I20260805 23:10:20.101325 28334 consensus_queue.cc:260] T 906bbc2a7f06406da9a9094adfea8764 P 3fa2896d2f224c69b1bb196eb4619a17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2b8471244eb4bb4825f255d21e3cdba" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 34855 } } peers { permanent_uuid: "3fa2896d2f224c69b1bb196eb4619a17" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 42229 } } peers { permanent_uuid: "ee3ec60b51b44ca995c23c3908f540b5" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38893 } }
I20260805 23:10:20.101568 28333 ts_tablet_manager.cc:1428] T 906bbc2a7f06406da9a9094adfea8764 P a2b8471244eb4bb4825f255d21e3cdba: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260805 23:10:20.103006 28334 ts_tablet_manager.cc:1428] T 906bbc2a7f06406da9a9094adfea8764 P 3fa2896d2f224c69b1bb196eb4619a17: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260805 23:10:20.105008 28335 tablet_bootstrap.cc:492] T 906bbc2a7f06406da9a9094adfea8764 P ee3ec60b51b44ca995c23c3908f540b5: No bootstrap required, opened a new log
I20260805 23:10:20.105119 28335 ts_tablet_manager.cc:1397] T 906bbc2a7f06406da9a9094adfea8764 P ee3ec60b51b44ca995c23c3908f540b5: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20260805 23:10:20.107859 28335 raft_consensus.cc:348] T 906bbc2a7f06406da9a9094adfea8764 P ee3ec60b51b44ca995c23c3908f540b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2b8471244eb4bb4825f255d21e3cdba" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 34855 } } peers { permanent_uuid: "3fa2896d2f224c69b1bb196eb4619a17" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 42229 } } peers { permanent_uuid: "ee3ec60b51b44ca995c23c3908f540b5" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38893 } }
I20260805 23:10:20.108022 28335 raft_consensus.cc:374] T 906bbc2a7f06406da9a9094adfea8764 P ee3ec60b51b44ca995c23c3908f540b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:10:20.108060 28335 raft_consensus.cc:729] T 906bbc2a7f06406da9a9094adfea8764 P ee3ec60b51b44ca995c23c3908f540b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee3ec60b51b44ca995c23c3908f540b5, State: Initialized, Role: FOLLOWER
I20260805 23:10:20.108197 28335 consensus_queue.cc:260] T 906bbc2a7f06406da9a9094adfea8764 P ee3ec60b51b44ca995c23c3908f540b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2b8471244eb4bb4825f255d21e3cdba" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 34855 } } peers { permanent_uuid: "3fa2896d2f224c69b1bb196eb4619a17" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 42229 } } peers { permanent_uuid: "ee3ec60b51b44ca995c23c3908f540b5" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38893 } }
I20260805 23:10:20.108470 28335 ts_tablet_manager.cc:1428] T 906bbc2a7f06406da9a9094adfea8764 P ee3ec60b51b44ca995c23c3908f540b5: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260805 23:10:20.154037 28344 raft_consensus.cc:482] T 906bbc2a7f06406da9a9094adfea8764 P ee3ec60b51b44ca995c23c3908f540b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 23:10:20.154202 28344 raft_consensus.cc:504] T 906bbc2a7f06406da9a9094adfea8764 P ee3ec60b51b44ca995c23c3908f540b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2b8471244eb4bb4825f255d21e3cdba" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 34855 } } peers { permanent_uuid: "3fa2896d2f224c69b1bb196eb4619a17" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 42229 } } peers { permanent_uuid: "ee3ec60b51b44ca995c23c3908f540b5" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38893 } }
I20260805 23:10:20.154748 28344 leader_election.cc:288] T 906bbc2a7f06406da9a9094adfea8764 P ee3ec60b51b44ca995c23c3908f540b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2b8471244eb4bb4825f255d21e3cdba (127.25.92.193:34855), 3fa2896d2f224c69b1bb196eb4619a17 (127.25.92.194:42229)
I20260805 23:10:20.155189 27888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "906bbc2a7f06406da9a9094adfea8764" candidate_uuid: "ee3ec60b51b44ca995c23c3908f540b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b8471244eb4bb4825f255d21e3cdba" is_pre_election: true
I20260805 23:10:20.155265 28030 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "906bbc2a7f06406da9a9094adfea8764" candidate_uuid: "ee3ec60b51b44ca995c23c3908f540b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fa2896d2f224c69b1bb196eb4619a17" is_pre_election: true
I20260805 23:10:20.155359 27888 raft_consensus.cc:2454] T 906bbc2a7f06406da9a9094adfea8764 P a2b8471244eb4bb4825f255d21e3cdba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee3ec60b51b44ca995c23c3908f540b5 in term 0.
I20260805 23:10:20.155398 28030 raft_consensus.cc:2454] T 906bbc2a7f06406da9a9094adfea8764 P 3fa2896d2f224c69b1bb196eb4619a17 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee3ec60b51b44ca995c23c3908f540b5 in term 0.
I20260805 23:10:20.155692 28102 leader_election.cc:302] T 906bbc2a7f06406da9a9094adfea8764 P ee3ec60b51b44ca995c23c3908f540b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fa2896d2f224c69b1bb196eb4619a17, ee3ec60b51b44ca995c23c3908f540b5; no voters:
I20260805 23:10:20.155882 28344 raft_consensus.cc:2781] T 906bbc2a7f06406da9a9094adfea8764 P ee3ec60b51b44ca995c23c3908f540b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260805 23:10:20.155947 28344 raft_consensus.cc:482] T 906bbc2a7f06406da9a9094adfea8764 P ee3ec60b51b44ca995c23c3908f540b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260805 23:10:20.155989 28344 raft_consensus.cc:3037] T 906bbc2a7f06406da9a9094adfea8764 P ee3ec60b51b44ca995c23c3908f540b5 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:10:20.157059 28344 raft_consensus.cc:504] T 906bbc2a7f06406da9a9094adfea8764 P ee3ec60b51b44ca995c23c3908f540b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2b8471244eb4bb4825f255d21e3cdba" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 34855 } } peers { permanent_uuid: "3fa2896d2f224c69b1bb196eb4619a17" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 42229 } } peers { permanent_uuid: "ee3ec60b51b44ca995c23c3908f540b5" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38893 } }
I20260805 23:10:20.157442 28344 leader_election.cc:288] T 906bbc2a7f06406da9a9094adfea8764 P ee3ec60b51b44ca995c23c3908f540b5 [CANDIDATE]: Term 1 election: Requested vote from peers a2b8471244eb4bb4825f255d21e3cdba (127.25.92.193:34855), 3fa2896d2f224c69b1bb196eb4619a17 (127.25.92.194:42229)
I20260805 23:10:20.157802 27888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "906bbc2a7f06406da9a9094adfea8764" candidate_uuid: "ee3ec60b51b44ca995c23c3908f540b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2b8471244eb4bb4825f255d21e3cdba"
I20260805 23:10:20.157940 27888 raft_consensus.cc:3037] T 906bbc2a7f06406da9a9094adfea8764 P a2b8471244eb4bb4825f255d21e3cdba [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:10:20.157934 28030 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "906bbc2a7f06406da9a9094adfea8764" candidate_uuid: "ee3ec60b51b44ca995c23c3908f540b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fa2896d2f224c69b1bb196eb4619a17"
I20260805 23:10:20.158079 28030 raft_consensus.cc:3037] T 906bbc2a7f06406da9a9094adfea8764 P 3fa2896d2f224c69b1bb196eb4619a17 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:10:20.158898 27888 raft_consensus.cc:2454] T 906bbc2a7f06406da9a9094adfea8764 P a2b8471244eb4bb4825f255d21e3cdba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee3ec60b51b44ca995c23c3908f540b5 in term 1.
I20260805 23:10:20.159268 28103 leader_election.cc:302] T 906bbc2a7f06406da9a9094adfea8764 P ee3ec60b51b44ca995c23c3908f540b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2b8471244eb4bb4825f255d21e3cdba, ee3ec60b51b44ca995c23c3908f540b5; no voters:
I20260805 23:10:20.159268 28030 raft_consensus.cc:2454] T 906bbc2a7f06406da9a9094adfea8764 P 3fa2896d2f224c69b1bb196eb4619a17 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee3ec60b51b44ca995c23c3908f540b5 in term 1.
I20260805 23:10:20.159468 28344 raft_consensus.cc:2781] T 906bbc2a7f06406da9a9094adfea8764 P ee3ec60b51b44ca995c23c3908f540b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:10:20.159569 28344 raft_consensus.cc:686] T 906bbc2a7f06406da9a9094adfea8764 P ee3ec60b51b44ca995c23c3908f540b5 [term 1 LEADER]: Becoming Leader. State: Replica: ee3ec60b51b44ca995c23c3908f540b5, State: Running, Role: LEADER
I20260805 23:10:20.159722 28344 consensus_queue.cc:237] T 906bbc2a7f06406da9a9094adfea8764 P ee3ec60b51b44ca995c23c3908f540b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2b8471244eb4bb4825f255d21e3cdba" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 34855 } } peers { permanent_uuid: "3fa2896d2f224c69b1bb196eb4619a17" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 42229 } } peers { permanent_uuid: "ee3ec60b51b44ca995c23c3908f540b5" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38893 } }
I20260805 23:10:20.161621 27724 catalog_manager.cc:5495] T 906bbc2a7f06406da9a9094adfea8764 P ee3ec60b51b44ca995c23c3908f540b5 reported cstate change: term changed from 0 to 1, leader changed from <none> to ee3ec60b51b44ca995c23c3908f540b5 (127.25.92.195). New cstate: current_term: 1 leader_uuid: "ee3ec60b51b44ca995c23c3908f540b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2b8471244eb4bb4825f255d21e3cdba" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3fa2896d2f224c69b1bb196eb4619a17" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 42229 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee3ec60b51b44ca995c23c3908f540b5" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38893 } health_report { overall_health: HEALTHY } } }
I20260805 23:10:20.195670 27888 raft_consensus.cc:1261] T 906bbc2a7f06406da9a9094adfea8764 P a2b8471244eb4bb4825f255d21e3cdba [term 1 FOLLOWER]: Refusing update from remote peer ee3ec60b51b44ca995c23c3908f540b5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:10:20.195744 28030 raft_consensus.cc:1261] T 906bbc2a7f06406da9a9094adfea8764 P 3fa2896d2f224c69b1bb196eb4619a17 [term 1 FOLLOWER]: Refusing update from remote peer ee3ec60b51b44ca995c23c3908f540b5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:10:20.196189 28344 consensus_queue.cc:1035] T 906bbc2a7f06406da9a9094adfea8764 P ee3ec60b51b44ca995c23c3908f540b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2b8471244eb4bb4825f255d21e3cdba" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:10:20.196421 28339 consensus_queue.cc:1035] T 906bbc2a7f06406da9a9094adfea8764 P ee3ec60b51b44ca995c23c3908f540b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fa2896d2f224c69b1bb196eb4619a17" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 42229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260805 23:10:20.214022 28303 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260805 23:10:21.780143 27724 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52142:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:10:21.780308 27724 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52142:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:10:21.783339 27724 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ea219c710c774b9d9e270e6fad216a05: Sending DeleteTablet for 3 replicas of tablet 906bbc2a7f06406da9a9094adfea8764
I20260805 23:10:21.784049 27867 tablet_service.cc:1514] Processing DeleteTablet for tablet 906bbc2a7f06406da9a9094adfea8764 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:10:21 UTC) from {username='kudu', principal='kudu/127.25.92.254@KRBTEST.COM'} at 127.0.0.1:49550
I20260805 23:10:21.784209 28152 tablet_service.cc:1514] Processing DeleteTablet for tablet 906bbc2a7f06406da9a9094adfea8764 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:10:21 UTC) from {username='kudu', principal='kudu/127.25.92.254@KRBTEST.COM'} at 127.0.0.1:46668
I20260805 23:10:21.784224 28010 tablet_service.cc:1514] Processing DeleteTablet for tablet 906bbc2a7f06406da9a9094adfea8764 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:10:21 UTC) from {username='kudu', principal='kudu/127.25.92.254@KRBTEST.COM'} at 127.0.0.1:33210
I20260805 23:10:21.784539 28386 tablet_replica.cc:330] T 906bbc2a7f06406da9a9094adfea8764 P a2b8471244eb4bb4825f255d21e3cdba: stopping tablet replica
I20260805 23:10:21.784569 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 27802
I20260805 23:10:21.789234 28388 tablet_replica.cc:330] T 906bbc2a7f06406da9a9094adfea8764 P 3fa2896d2f224c69b1bb196eb4619a17: stopping tablet replica
I20260805 23:10:21.789429 28388 raft_consensus.cc:2229] T 906bbc2a7f06406da9a9094adfea8764 P 3fa2896d2f224c69b1bb196eb4619a17 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 23:10:21.789726 28388 raft_consensus.cc:2258] T 906bbc2a7f06406da9a9094adfea8764 P 3fa2896d2f224c69b1bb196eb4619a17 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:10:21.789840 28387 tablet_replica.cc:330] T 906bbc2a7f06406da9a9094adfea8764 P ee3ec60b51b44ca995c23c3908f540b5: stopping tablet replica
I20260805 23:10:21.790001 28387 raft_consensus.cc:2229] T 906bbc2a7f06406da9a9094adfea8764 P ee3ec60b51b44ca995c23c3908f540b5 [term 1 LEADER]: Raft consensus shutting down.
I20260805 23:10:21.790285 28387 raft_consensus.cc:2258] T 906bbc2a7f06406da9a9094adfea8764 P ee3ec60b51b44ca995c23c3908f540b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:10:21.790594 28388 ts_tablet_manager.cc:1905] T 906bbc2a7f06406da9a9094adfea8764 P 3fa2896d2f224c69b1bb196eb4619a17: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:10:21.790972 28387 ts_tablet_manager.cc:1905] T 906bbc2a7f06406da9a9094adfea8764 P ee3ec60b51b44ca995c23c3908f540b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:10:21.793895 28388 ts_tablet_manager.cc:1918] T 906bbc2a7f06406da9a9094adfea8764 P 3fa2896d2f224c69b1bb196eb4619a17: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260805 23:10:21.793895 28387 ts_tablet_manager.cc:1918] T 906bbc2a7f06406da9a9094adfea8764 P ee3ec60b51b44ca995c23c3908f540b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260805 23:10:21.793984 28387 log.cc:1196] T 906bbc2a7f06406da9a9094adfea8764 P ee3ec60b51b44ca995c23c3908f540b5: Deleting WAL directory at /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/minicluster-data/ts-2/wal/wals/906bbc2a7f06406da9a9094adfea8764
I20260805 23:10:21.793998 28388 log.cc:1196] T 906bbc2a7f06406da9a9094adfea8764 P 3fa2896d2f224c69b1bb196eb4619a17: Deleting WAL directory at /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971406251336-25971-0/minicluster-data/ts-1/wal/wals/906bbc2a7f06406da9a9094adfea8764
I20260805 23:10:21.794299 28388 ts_tablet_manager.cc:1939] T 906bbc2a7f06406da9a9094adfea8764 P 3fa2896d2f224c69b1bb196eb4619a17: Deleting consensus metadata
I20260805 23:10:21.794312 28387 ts_tablet_manager.cc:1939] T 906bbc2a7f06406da9a9094adfea8764 P ee3ec60b51b44ca995c23c3908f540b5: Deleting consensus metadata
I20260805 23:10:21.795233 27711 catalog_manager.cc:4841] TS 3fa2896d2f224c69b1bb196eb4619a17 (127.25.92.194:42229): tablet 906bbc2a7f06406da9a9094adfea8764 (table test-table [id=588d93a47eb142daa51320ded03c94d9]) successfully deleted
I20260805 23:10:21.795433 27711 catalog_manager.cc:4841] TS ee3ec60b51b44ca995c23c3908f540b5 (127.25.92.195:38893): tablet 906bbc2a7f06406da9a9094adfea8764 (table test-table [id=588d93a47eb142daa51320ded03c94d9]) successfully deleted
W20260805 23:10:21.795490 27712 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.25.92.193:34855 (error 108)
I20260805 23:10:21.795547 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 27944
I20260805 23:10:21.804416 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 28086
I20260805 23:10:21.806948 28390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:21.805620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.193:34855 (local address 127.0.0.1:49564)
0805 23:10:21.806740 (+ 1120us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:21.806835 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.193:34855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1028,"thread_start_us":112,"threads_started":1}
W20260805 23:10:21.807154 27712 catalog_manager.cc:4584] TS a2b8471244eb4bb4825f255d21e3cdba (127.25.92.193:34855): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 906bbc2a7f06406da9a9094adfea8764: Network error: Client connection negotiation failed: client connection to 127.25.92.193:34855: connect: Connection refused (error 111)
I20260805 23:10:21.812407 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 27695
2026-08-05T23:10:21Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6488 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971406251336-25971-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 23:10:21 dist-test-slave-ss4n krb5kdc[28393](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 23:10:21 dist-test-slave-ss4n krb5kdc[28393](info): set up 2 sockets
Aug 05 23:10:21 dist-test-slave-ss4n krb5kdc[28393](info): commencing operation
krb5kdc: starting...
W20260805 23:10:22.955813 25971 mini_kdc.cc:121] Time spent starting KDC: real 1.117s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971406251336-25971-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971406251336-25971-0/krb5kdc/test-user.keytab.
Aug 05 23:10:23 dist-test-slave-ss4n krb5kdc[28393](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971423, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T23:10:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:10:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.92.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.92.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.92.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab.
Entry for principal kudu/127.25.92.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab.
I20260805 23:10:23.092700 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971406251336-25971-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971406251336-25971-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971406251336-25971-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.254:38089
--webserver_interface=127.25.92.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab
--principal=kudu/127.25.92.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:41493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.92.254:38089
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:10:23.208113 28406 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:10:23.208444 28406 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:10:23.208523 28406 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:10:23.212667 28406 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:10:23.212780 28406 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:10:23.212819 28406 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:10:23.212859 28406 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:10:23.212894 28406 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:10:23.218559 28406 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:41493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971406251336-25971-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971406251336-25971-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.92.254:38089
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab
--principal=kudu/127.25.92.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.254:38089
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.92.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28406
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971406251336-25971-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:10:23.219713 28406 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:10:23.220752 28406 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260805 23:10:23.230373 28406 server_base.cc:1034] running on GCE node
W20260805 23:10:23.230392 28411 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:23.230392 28412 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:23.230621 28414 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:10:23.231344 28406 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:10:23.232553 28406 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:10:23.233752 28406 hybrid_clock.cc:648] HybridClock initialized: now 1785971423233739 us; error 27 us; skew 500 ppm
Aug 05 23:10:23 dist-test-slave-ss4n krb5kdc[28393](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971423, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:10:23.236650 28406 init.cc:377] Logged in from keytab as kudu/127.25.92.254@KRBTEST.COM (short username kudu)
I20260805 23:10:23.237717 28406 webserver.cc:460] Webserver started at http://127.25.92.254:35217/ using document root <none> and password file <none>
I20260805 23:10:23.238353 28406 fs_manager.cc:359] Metadata directory not provided
I20260805 23:10:23.238415 28406 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:10:23.238585 28406 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:10:23.240710 28406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971406251336-25971-0/minicluster-data/master-0/data/instance:
uuid: "c9168058d5d048bea20b48a125bf4729"
format_stamp: "Formatted at 2026-08-05 23:10:23 on dist-test-slave-ss4n"
server_key: "a2e4089293b5436527240a3abecda27e"
server_key_iv: "5941812572155890a9ec4f1e13bb2f27"
server_key_version: "encryptionkey@0"
I20260805 23:10:23.241243 28406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971406251336-25971-0/minicluster-data/master-0/wal/instance:
uuid: "c9168058d5d048bea20b48a125bf4729"
format_stamp: "Formatted at 2026-08-05 23:10:23 on dist-test-slave-ss4n"
server_key: "a2e4089293b5436527240a3abecda27e"
server_key_iv: "5941812572155890a9ec4f1e13bb2f27"
server_key_version: "encryptionkey@0"
I20260805 23:10:23.245177 28406 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260805 23:10:23.248046 28421 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:23.249667 28406 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:10:23.249787 28406 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971406251336-25971-0/minicluster-data/master-0/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971406251336-25971-0/minicluster-data/master-0/wal
uuid: "c9168058d5d048bea20b48a125bf4729"
format_stamp: "Formatted at 2026-08-05 23:10:23 on dist-test-slave-ss4n"
server_key: "a2e4089293b5436527240a3abecda27e"
server_key_iv: "5941812572155890a9ec4f1e13bb2f27"
server_key_version: "encryptionkey@0"
I20260805 23:10:23.249888 28406 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971406251336-25971-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971406251336-25971-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971406251336-25971-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:10:23.267665 28406 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:10:23.271342 28406 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:10:23.271517 28406 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:10:23.279366 28406 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.254:38089
I20260805 23:10:23.279394 28473 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.254:38089 every 8 connection(s)
I20260805 23:10:23.280434 28406 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
I20260805 23:10:23.283310 28474 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:10:23.289295 28474 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9168058d5d048bea20b48a125bf4729: Bootstrap starting.
I20260805 23:10:23.290338 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 28406
I20260805 23:10:23.290457 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971406251336-25971-0/minicluster-data/master-0/wal/instance
I20260805 23:10:23.290763 25971 external_mini_cluster.cc:1428] Setting key 88ce22b8b99f694f0d0e201094e78854
I20260805 23:10:23.292579 28474 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c9168058d5d048bea20b48a125bf4729: Neither blocks nor log segments found. Creating new log.
I20260805 23:10:23.293360 28474 log.cc:824] T 00000000000000000000000000000000 P c9168058d5d048bea20b48a125bf4729: Log is configured to *not* fsync() on all Append() calls
I20260805 23:10:23.295699 28474 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9168058d5d048bea20b48a125bf4729: No bootstrap required, opened a new log
Aug 05 23:10:23 dist-test-slave-ss4n krb5kdc[28393](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971423, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM
I20260805 23:10:23.301189 28474 raft_consensus.cc:348] T 00000000000000000000000000000000 P c9168058d5d048bea20b48a125bf4729 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9168058d5d048bea20b48a125bf4729" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 38089 } }
I20260805 23:10:23.301527 28474 raft_consensus.cc:374] T 00000000000000000000000000000000 P c9168058d5d048bea20b48a125bf4729 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:10:23.301610 28474 raft_consensus.cc:729] T 00000000000000000000000000000000 P c9168058d5d048bea20b48a125bf4729 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9168058d5d048bea20b48a125bf4729, State: Initialized, Role: FOLLOWER
I20260805 23:10:23.302145 28474 consensus_queue.cc:260] T 00000000000000000000000000000000 P c9168058d5d048bea20b48a125bf4729 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9168058d5d048bea20b48a125bf4729" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 38089 } }
I20260805 23:10:23.302348 28474 raft_consensus.cc:388] T 00000000000000000000000000000000 P c9168058d5d048bea20b48a125bf4729 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:10:23.302445 28474 raft_consensus.cc:482] T 00000000000000000000000000000000 P c9168058d5d048bea20b48a125bf4729 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:10:23.302613 28474 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c9168058d5d048bea20b48a125bf4729 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:10:23.303673 28474 raft_consensus.cc:504] T 00000000000000000000000000000000 P c9168058d5d048bea20b48a125bf4729 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9168058d5d048bea20b48a125bf4729" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 38089 } }
I20260805 23:10:23.304044 28474 leader_election.cc:302] T 00000000000000000000000000000000 P c9168058d5d048bea20b48a125bf4729 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c9168058d5d048bea20b48a125bf4729; no voters:
I20260805 23:10:23.304436 28474 leader_election.cc:288] T 00000000000000000000000000000000 P c9168058d5d048bea20b48a125bf4729 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 23:10:23.304593 28479 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c9168058d5d048bea20b48a125bf4729 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:10:23.304901 28479 raft_consensus.cc:686] T 00000000000000000000000000000000 P c9168058d5d048bea20b48a125bf4729 [term 1 LEADER]: Becoming Leader. State: Replica: c9168058d5d048bea20b48a125bf4729, State: Running, Role: LEADER
I20260805 23:10:23.305241 28479 consensus_queue.cc:237] T 00000000000000000000000000000000 P c9168058d5d048bea20b48a125bf4729 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9168058d5d048bea20b48a125bf4729" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 38089 } }
I20260805 23:10:23.305979 28474 sys_catalog.cc:564] T 00000000000000000000000000000000 P c9168058d5d048bea20b48a125bf4729 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:10:23.307045 28480 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9168058d5d048bea20b48a125bf4729 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c9168058d5d048bea20b48a125bf4729" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9168058d5d048bea20b48a125bf4729" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 38089 } } }
I20260805 23:10:23.307186 28480 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9168058d5d048bea20b48a125bf4729 [sys.catalog]: This master's current role is: LEADER
I20260805 23:10:23.307519 28481 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9168058d5d048bea20b48a125bf4729 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c9168058d5d048bea20b48a125bf4729. Latest consensus state: current_term: 1 leader_uuid: "c9168058d5d048bea20b48a125bf4729" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9168058d5d048bea20b48a125bf4729" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 38089 } } }
I20260805 23:10:23.307618 28481 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9168058d5d048bea20b48a125bf4729 [sys.catalog]: This master's current role is: LEADER
I20260805 23:10:23.308920 28477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:23.292170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35742 (local address 127.25.92.254:38089)
0805 23:10:23.292741 (+ 571us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:23.292773 (+ 32us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:23.292819 (+ 46us) server_negotiation.cc:407] Connection header received
0805 23:10:23.293701 (+ 882us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:23.293723 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:23.294110 (+ 387us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:23.294488 (+ 378us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:23.295350 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:23.296491 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:23.298114 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:23.298729 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:23.298865 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:23.302187 (+ 3322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:23.302193 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:23.302208 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:23.302243 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:23.304364 (+ 2121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:23.305108 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:23.305114 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:23.305120 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:23.305269 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:23.305621 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:23.305624 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:23.305626 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:23.306149 (+ 523us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:23.306363 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:23.306705 (+ 342us) server_negotiation.cc:299] Negotiation successful
0805 23:10:23.307093 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":182,"threads_started":1}
W20260805 23:10:23.310952 28494 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c9168058d5d048bea20b48a125bf4729: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260805 23:10:23.311057 28494 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260805 23:10:23.311131 28488 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:10:23.311990 28488 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:10:23.318270 28488 catalog_manager.cc:1347] Generated new cluster ID: 54641a78b3e74c54a0366ef27b2a8fdd
I20260805 23:10:23.318374 28488 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:10:23.353185 28488 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:10:23.354414 28488 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:10:23.362902 28488 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c9168058d5d048bea20b48a125bf4729: Generated new TSK 0
I20260805 23:10:23.363844 28488 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.92.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.92.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.92.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.193.keytab.
Entry for principal kudu/127.25.92.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.193.keytab.
I20260805 23:10:23.402304 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971406251336-25971-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971406251336-25971-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971406251336-25971-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971406251336-25971-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.92.193:0
--local_ip_for_outbound_sockets=127.25.92.193
--webserver_interface=127.25.92.193
--webserver_port=0
--tserver_master_addrs=127.25.92.254:38089
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.193.keytab
--principal=kudu/127.25.92.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:41493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:10:23.516278 28500 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:10:23.516562 28500 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:10:23.516607 28500 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:10:23.520749 28500 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:10:23.520824 28500 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:10:23.520886 28500 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.92.193
I20260805 23:10:23.526360 28500 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:41493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971406251336-25971-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971406251336-25971-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.193.keytab
--principal=kudu/127.25.92.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.92.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971406251336-25971-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.92.193
--webserver_port=0
--tserver_master_addrs=127.25.92.254:38089
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28500
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.92.193
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971406251336-25971-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:10:23.527405 28500 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:10:23.528453 28500 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:10:23.538158 28508 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:23.538189 28506 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:10:23.538183 28500 server_base.cc:1034] running on GCE node
W20260805 23:10:23.538161 28505 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:10:23.538990 28500 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:10:23.539666 28500 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:10:23.540841 28500 hybrid_clock.cc:648] HybridClock initialized: now 1785971423540824 us; error 34 us; skew 500 ppm
Aug 05 23:10:23 dist-test-slave-ss4n krb5kdc[28393](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971423, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:10:23.543936 28500 init.cc:377] Logged in from keytab as kudu/127.25.92.193@KRBTEST.COM (short username kudu)
I20260805 23:10:23.544876 28500 webserver.cc:460] Webserver started at http://127.25.92.193:39309/ using document root <none> and password file <none>
I20260805 23:10:23.545408 28500 fs_manager.cc:359] Metadata directory not provided
I20260805 23:10:23.545467 28500 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:10:23.545723 28500 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:10:23.547784 28500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971406251336-25971-0/minicluster-data/ts-0/data/instance:
uuid: "79ca2eae3d2347a981a9d5596ea186b8"
format_stamp: "Formatted at 2026-08-05 23:10:23 on dist-test-slave-ss4n"
server_key: "81e4ebb348968b31c475acad261a1a17"
server_key_iv: "3db5688511d5432f0ce903084c04f676"
server_key_version: "encryptionkey@0"
I20260805 23:10:23.548292 28500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971406251336-25971-0/minicluster-data/ts-0/wal/instance:
uuid: "79ca2eae3d2347a981a9d5596ea186b8"
format_stamp: "Formatted at 2026-08-05 23:10:23 on dist-test-slave-ss4n"
server_key: "81e4ebb348968b31c475acad261a1a17"
server_key_iv: "3db5688511d5432f0ce903084c04f676"
server_key_version: "encryptionkey@0"
I20260805 23:10:23.552357 28500 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260805 23:10:23.555343 28515 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:23.556748 28500 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260805 23:10:23.556912 28500 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971406251336-25971-0/minicluster-data/ts-0/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971406251336-25971-0/minicluster-data/ts-0/wal
uuid: "79ca2eae3d2347a981a9d5596ea186b8"
format_stamp: "Formatted at 2026-08-05 23:10:23 on dist-test-slave-ss4n"
server_key: "81e4ebb348968b31c475acad261a1a17"
server_key_iv: "3db5688511d5432f0ce903084c04f676"
server_key_version: "encryptionkey@0"
I20260805 23:10:23.557053 28500 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971406251336-25971-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971406251336-25971-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971406251336-25971-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:10:23.584120 28500 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:10:23.587785 28500 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:10:23.588016 28500 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:10:23.588686 28500 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:10:23.589715 28500 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:10:23.589783 28500 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:23.589857 28500 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:10:23.589902 28500 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:23.599445 28500 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.193:36743
I20260805 23:10:23.599462 28630 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.193:36743 every 8 connection(s)
I20260805 23:10:23.600389 28500 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971406251336-25971-0/minicluster-data/ts-0/data/info.pb
I20260805 23:10:23.609745 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 28500
I20260805 23:10:23.609890 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971406251336-25971-0/minicluster-data/ts-0/wal/instance
I20260805 23:10:23.610222 25971 external_mini_cluster.cc:1428] Setting key abcec19962bca11bee5f86870c30303d
Aug 05 23:10:23 dist-test-slave-ss4n krb5kdc[28393](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971423, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.193@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM
I20260805 23:10:23.615752 28477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:23.602335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.193:48639 (local address 127.25.92.254:38089)
0805 23:10:23.602523 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:23.602529 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:23.603585 (+ 1056us) server_negotiation.cc:407] Connection header received
0805 23:10:23.604419 (+ 834us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:23.604422 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:23.604472 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:23.604556 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:23.605881 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:23.606460 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:23.607859 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:23.608100 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:23.608199 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:23.611211 (+ 3012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:23.611214 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:23.611216 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:23.611242 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:23.613129 (+ 1887us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:23.613750 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:23.613757 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:23.613762 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:23.613873 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:23.614346 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:23.614348 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:23.614350 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:23.614587 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:23.614676 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:23.615471 (+ 795us) server_negotiation.cc:299] Negotiation successful
0805 23:10:23.615618 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260805 23:10:23.616920 28633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:23.602768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:38089 (local address 127.25.92.193:48639)
0805 23:10:23.603436 (+ 668us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:23.603477 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:23.604237 (+ 760us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:23.604758 (+ 521us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:23.604764 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:23.605117 (+ 353us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:23.605702 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:23.605719 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:23.606646 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:23.606654 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:23.607731 (+ 1077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:23.607740 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:23.608400 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:23.608405 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:23.608553 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:23.609218 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:10:23.609234 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:23.610993 (+ 1759us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:10:23.613272 (+ 2279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:23.613278 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:23.613292 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:23.613644 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:23.613991 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:23.613996 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:23.613999 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:23.614229 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:23.614705 (+ 476us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:10:23.614713 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:10:23.615132 (+ 419us) client_negotiation.cc:769] Sending connection context
0805 23:10:23.615366 (+ 234us) client_negotiation.cc:240] Negotiation successful
0805 23:10:23.615748 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":114,"threads_started":1}
I20260805 23:10:23.618377 28631 heartbeater.cc:347] Connected to a master server at 127.25.92.254:38089
I20260805 23:10:23.618698 28631 heartbeater.cc:464] Registering TS with master...
I20260805 23:10:23.619238 28631 heartbeater.cc:511] Master 127.25.92.254:38089 requested a full tablet report, sending...
I20260805 23:10:23.620913 28438 ts_manager.cc:194] Registered new tserver with Master: 79ca2eae3d2347a981a9d5596ea186b8 (127.25.92.193:36743)
WARNING: no policy specified for kudu/127.25.92.194@KRBTEST.COM; defaulting to no policy
I20260805 23:10:23.622253 28438 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.92.193@KRBTEST.COM'} at 127.25.92.193:48639
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.92.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.92.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.194.keytab.
Entry for principal kudu/127.25.92.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.194.keytab.
I20260805 23:10:23.641986 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971406251336-25971-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971406251336-25971-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971406251336-25971-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971406251336-25971-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.92.194:0
--local_ip_for_outbound_sockets=127.25.92.194
--webserver_interface=127.25.92.194
--webserver_port=0
--tserver_master_addrs=127.25.92.254:38089
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.194.keytab
--principal=kudu/127.25.92.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:41493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:10:23.757866 28636 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:10:23.758140 28636 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:10:23.758220 28636 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:10:23.762575 28636 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:10:23.762648 28636 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:10:23.762737 28636 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.92.194
I20260805 23:10:23.768280 28636 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:41493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971406251336-25971-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971406251336-25971-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.194.keytab
--principal=kudu/127.25.92.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.92.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971406251336-25971-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.92.194
--webserver_port=0
--tserver_master_addrs=127.25.92.254:38089
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28636
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.92.194
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971406251336-25971-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:10:23.769405 28636 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:10:23.770625 28636 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:10:23.780654 28641 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:23.780603 28642 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:23.780620 28644 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:10:23.780925 28636 server_base.cc:1034] running on GCE node
I20260805 23:10:23.781368 28636 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:10:23.782081 28636 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:10:23.783259 28636 hybrid_clock.cc:648] HybridClock initialized: now 1785971423783239 us; error 30 us; skew 500 ppm
Aug 05 23:10:23 dist-test-slave-ss4n krb5kdc[28393](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971423, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:10:23.786305 28636 init.cc:377] Logged in from keytab as kudu/127.25.92.194@KRBTEST.COM (short username kudu)
I20260805 23:10:23.787186 28636 webserver.cc:460] Webserver started at http://127.25.92.194:38053/ using document root <none> and password file <none>
I20260805 23:10:23.787763 28636 fs_manager.cc:359] Metadata directory not provided
I20260805 23:10:23.787853 28636 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:10:23.788089 28636 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:10:23.790242 28636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971406251336-25971-0/minicluster-data/ts-1/data/instance:
uuid: "add22f11f0884892a24e8b9e571688d9"
format_stamp: "Formatted at 2026-08-05 23:10:23 on dist-test-slave-ss4n"
server_key: "869054ccc28b92f7c6563f84dc87df4a"
server_key_iv: "473abc3149bdaa13a7d585730c2a7fa5"
server_key_version: "encryptionkey@0"
I20260805 23:10:23.790820 28636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971406251336-25971-0/minicluster-data/ts-1/wal/instance:
uuid: "add22f11f0884892a24e8b9e571688d9"
format_stamp: "Formatted at 2026-08-05 23:10:23 on dist-test-slave-ss4n"
server_key: "869054ccc28b92f7c6563f84dc87df4a"
server_key_iv: "473abc3149bdaa13a7d585730c2a7fa5"
server_key_version: "encryptionkey@0"
I20260805 23:10:23.794641 28636 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260805 23:10:23.797187 28651 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:23.798492 28636 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260805 23:10:23.798640 28636 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971406251336-25971-0/minicluster-data/ts-1/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971406251336-25971-0/minicluster-data/ts-1/wal
uuid: "add22f11f0884892a24e8b9e571688d9"
format_stamp: "Formatted at 2026-08-05 23:10:23 on dist-test-slave-ss4n"
server_key: "869054ccc28b92f7c6563f84dc87df4a"
server_key_iv: "473abc3149bdaa13a7d585730c2a7fa5"
server_key_version: "encryptionkey@0"
I20260805 23:10:23.798765 28636 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971406251336-25971-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971406251336-25971-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971406251336-25971-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:10:23.818893 28636 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:10:23.822517 28636 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:10:23.822749 28636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:10:23.823405 28636 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:10:23.824309 28636 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:10:23.824386 28636 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:23.824462 28636 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:10:23.824517 28636 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:23.834723 28636 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.194:37777
I20260805 23:10:23.834756 28765 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.194:37777 every 8 connection(s)
I20260805 23:10:23.835748 28636 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971406251336-25971-0/minicluster-data/ts-1/data/info.pb
I20260805 23:10:23.839334 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 28636
I20260805 23:10:23.839468 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971406251336-25971-0/minicluster-data/ts-1/wal/instance
I20260805 23:10:23.839797 25971 external_mini_cluster.cc:1428] Setting key acba7ee6e8a1b8ddec7c15aef6adf560
Aug 05 23:10:23 dist-test-slave-ss4n krb5kdc[28393](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971423, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.194@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.25.92.195@KRBTEST.COM; defaulting to no policy
I20260805 23:10:23.852401 28477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:23.838544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.194:47165 (local address 127.25.92.254:38089)
0805 23:10:23.838745 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:23.838749 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:23.839848 (+ 1099us) server_negotiation.cc:407] Connection header received
0805 23:10:23.840867 (+ 1019us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:23.840870 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:23.840917 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:23.841041 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:23.842429 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:23.843286 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:23.845223 (+ 1937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:23.845734 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:23.845867 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:23.848453 (+ 2586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:23.848458 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:23.848464 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:23.848512 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:23.850308 (+ 1796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:23.850861 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:23.850866 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:23.850869 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:23.850969 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:23.851378 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:23.851380 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:23.851381 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:23.851573 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:23.851653 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:23.852157 (+ 504us) server_negotiation.cc:299] Negotiation successful
0805 23:10:23.852274 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260805 23:10:23.853304 28768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:23.838900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:38089 (local address 127.25.92.194:47165)
0805 23:10:23.839672 (+ 772us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:23.839726 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:23.840653 (+ 927us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:23.841190 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:23.841197 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:23.841768 (+ 571us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:23.842278 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:23.842293 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:23.843497 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:23.843503 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:23.845068 (+ 1565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:23.845079 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:23.845897 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:23.845900 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:23.846046 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:23.846733 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:10:23.846750 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:23.848266 (+ 1516us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:10:23.850439 (+ 2173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:23.850444 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:23.850457 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:23.850747 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:23.851065 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:23.851067 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:23.851068 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:23.851279 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:23.851666 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:10:23.851672 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:10:23.851907 (+ 235us) client_negotiation.cc:769] Sending connection context
0805 23:10:23.852067 (+ 160us) client_negotiation.cc:240] Negotiation successful
0805 23:10:23.852349 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":564,"thread_start_us":167,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.92.195@KRBTEST.COM" created.
I20260805 23:10:23.854766 28766 heartbeater.cc:347] Connected to a master server at 127.25.92.254:38089
I20260805 23:10:23.855026 28766 heartbeater.cc:464] Registering TS with master...
I20260805 23:10:23.855641 28766 heartbeater.cc:511] Master 127.25.92.254:38089 requested a full tablet report, sending...
I20260805 23:10:23.856931 28438 ts_manager.cc:194] Registered new tserver with Master: add22f11f0884892a24e8b9e571688d9 (127.25.92.194:37777)
I20260805 23:10:23.857630 28438 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.92.194@KRBTEST.COM'} at 127.25.92.194:47165
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.92.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.195.keytab.
Entry for principal kudu/127.25.92.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.195.keytab.
I20260805 23:10:23.874500 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971406251336-25971-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971406251336-25971-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971406251336-25971-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971406251336-25971-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.92.195:0
--local_ip_for_outbound_sockets=127.25.92.195
--webserver_interface=127.25.92.195
--webserver_port=0
--tserver_master_addrs=127.25.92.254:38089
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.195.keytab
--principal=kudu/127.25.92.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:41493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:10:23.990085 28771 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:10:23.990391 28771 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:10:23.990470 28771 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:10:23.994480 28771 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:10:23.994588 28771 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:10:23.994688 28771 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.92.195
I20260805 23:10:24.000044 28771 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:41493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971406251336-25971-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971406251336-25971-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.195.keytab
--principal=kudu/127.25.92.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.92.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971406251336-25971-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.92.195
--webserver_port=0
--tserver_master_addrs=127.25.92.254:38089
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28771
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.92.195
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971406251336-25971-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:10:24.001194 28771 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:10:24.002311 28771 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:10:24.013029 28777 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:24.013029 28776 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:24.013031 28779 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:10:24.013338 28771 server_base.cc:1034] running on GCE node
I20260805 23:10:24.013880 28771 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:10:24.014564 28771 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:10:24.015763 28771 hybrid_clock.cc:648] HybridClock initialized: now 1785971424015730 us; error 44 us; skew 500 ppm
Aug 05 23:10:24 dist-test-slave-ss4n krb5kdc[28393](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971424, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:10:24.018752 28771 init.cc:377] Logged in from keytab as kudu/127.25.92.195@KRBTEST.COM (short username kudu)
I20260805 23:10:24.019651 28771 webserver.cc:460] Webserver started at http://127.25.92.195:38833/ using document root <none> and password file <none>
I20260805 23:10:24.020228 28771 fs_manager.cc:359] Metadata directory not provided
I20260805 23:10:24.020320 28771 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:10:24.020560 28771 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:10:24.022693 28771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971406251336-25971-0/minicluster-data/ts-2/data/instance:
uuid: "45df022ac2f44a4aba26097e857da4a3"
format_stamp: "Formatted at 2026-08-05 23:10:24 on dist-test-slave-ss4n"
server_key: "89b21a2f29be66eb37ad9c99705641c5"
server_key_iv: "6d29493467e9b3cf6059def56717fe12"
server_key_version: "encryptionkey@0"
I20260805 23:10:24.023257 28771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971406251336-25971-0/minicluster-data/ts-2/wal/instance:
uuid: "45df022ac2f44a4aba26097e857da4a3"
format_stamp: "Formatted at 2026-08-05 23:10:24 on dist-test-slave-ss4n"
server_key: "89b21a2f29be66eb37ad9c99705641c5"
server_key_iv: "6d29493467e9b3cf6059def56717fe12"
server_key_version: "encryptionkey@0"
I20260805 23:10:24.027073 28771 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260805 23:10:24.029629 28787 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:24.030846 28771 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260805 23:10:24.030987 28771 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971406251336-25971-0/minicluster-data/ts-2/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971406251336-25971-0/minicluster-data/ts-2/wal
uuid: "45df022ac2f44a4aba26097e857da4a3"
format_stamp: "Formatted at 2026-08-05 23:10:24 on dist-test-slave-ss4n"
server_key: "89b21a2f29be66eb37ad9c99705641c5"
server_key_iv: "6d29493467e9b3cf6059def56717fe12"
server_key_version: "encryptionkey@0"
I20260805 23:10:24.031111 28771 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971406251336-25971-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971406251336-25971-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971406251336-25971-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:10:24.044647 28771 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:10:24.047698 28771 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:10:24.047914 28771 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:10:24.048540 28771 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:10:24.049410 28771 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:10:24.049484 28771 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:24.049589 28771 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:10:24.049651 28771 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:24.059021 28771 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.195:44249
I20260805 23:10:24.059069 28901 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.195:44249 every 8 connection(s)
I20260805 23:10:24.059926 28771 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971406251336-25971-0/minicluster-data/ts-2/data/info.pb
I20260805 23:10:24.061678 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 28771
I20260805 23:10:24.061815 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971406251336-25971-0/minicluster-data/ts-2/wal/instance
I20260805 23:10:24.062093 25971 external_mini_cluster.cc:1428] Setting key a398300503944cc11d87b6b35a7c6bef
Aug 05 23:10:24 dist-test-slave-ss4n krb5kdc[28393](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971424, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.195@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM
I20260805 23:10:24.074215 28477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:24.062648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.195:51827 (local address 127.25.92.254:38089)
0805 23:10:24.062815 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:24.062820 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:24.063587 (+ 767us) server_negotiation.cc:407] Connection header received
0805 23:10:24.064680 (+ 1093us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:24.064683 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:24.064728 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:24.064796 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:24.065978 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:24.066516 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:24.067654 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:24.067873 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:24.067953 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:24.070454 (+ 2501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:24.070458 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:24.070461 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:24.070495 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:24.072063 (+ 1568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:24.072579 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:24.072583 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:24.072585 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:24.072671 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:24.073011 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:24.073015 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:24.073017 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:24.073221 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:24.073332 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:24.073898 (+ 566us) server_negotiation.cc:299] Negotiation successful
0805 23:10:24.074022 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260805 23:10:24.075105 28904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:24.062448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:38089 (local address 127.25.92.195:51827)
0805 23:10:24.063451 (+ 1003us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:24.063494 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:24.064470 (+ 976us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:24.064944 (+ 474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:24.064950 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:24.065309 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:24.065848 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:24.065861 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:24.066672 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:24.066674 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:24.067540 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:24.067549 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:24.067999 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:24.068002 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:24.068140 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:24.068747 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:10:24.068763 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:24.070283 (+ 1520us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:10:24.072192 (+ 1909us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:24.072198 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:24.072213 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:24.072472 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:24.072768 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:24.072772 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:24.072774 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:24.072915 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:24.073332 (+ 417us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:10:24.073340 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:10:24.073661 (+ 321us) client_negotiation.cc:769] Sending connection context
0805 23:10:24.073854 (+ 193us) client_negotiation.cc:240] Negotiation successful
0805 23:10:24.074182 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":784,"thread_start_us":146,"threads_started":1}
I20260805 23:10:24.076285 28902 heartbeater.cc:347] Connected to a master server at 127.25.92.254:38089
I20260805 23:10:24.076534 28902 heartbeater.cc:464] Registering TS with master...
I20260805 23:10:24.077014 28902 heartbeater.cc:511] Master 127.25.92.254:38089 requested a full tablet report, sending...
I20260805 23:10:24.078019 28438 ts_manager.cc:194] Registered new tserver with Master: 45df022ac2f44a4aba26097e857da4a3 (127.25.92.195:44249)
I20260805 23:10:24.078615 28438 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.92.195@KRBTEST.COM'} at 127.25.92.195:51827
I20260805 23:10:24.088563 25971 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260805 23:10:24.103577 28477 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:10:24.101907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35746 (local address 127.25.92.254:38089)
0805 23:10:24.102047 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:24.102051 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:24.102284 (+ 233us) server_negotiation.cc:407] Connection header received
0805 23:10:24.102394 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:24.102396 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:24.102515 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:24.102609 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:24.103397 (+ 788us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35746: BlockingRecv error: recv got EOF from 127.0.0.1:35746 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
I20260805 23:10:24.104817 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 28500
I20260805 23:10:24.114046 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 28636
I20260805 23:10:24.119375 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 28771
I20260805 23:10:24.127403 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 28406
2026-08-05T23:10:24Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2312 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785971406251336-25971-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 23:10:24 dist-test-slave-ss4n krb5kdc[28914](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 23:10:24 dist-test-slave-ss4n krb5kdc[28914](info): set up 2 sockets
Aug 05 23:10:24 dist-test-slave-ss4n krb5kdc[28914](info): commencing operation
krb5kdc: starting...
W20260805 23:10:25.240124 25971 mini_kdc.cc:121] Time spent starting KDC: real 1.089s user 0.007s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785971406251336-25971-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785971406251336-25971-0/krb5kdc/test-user.keytab.
Aug 05 23:10:25 dist-test-slave-ss4n krb5kdc[28914](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971425, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T23:10:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:10:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.92.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.92.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.92.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab.
Entry for principal kudu/127.25.92.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab.
I20260805 23:10:25.374013 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785971406251336-25971-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785971406251336-25971-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785971406251336-25971-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.254:33719
--webserver_interface=127.25.92.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab
--principal=kudu/127.25.92.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:43239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.92.254:33719
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:10:25.492578 28927 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:10:25.492888 28927 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:10:25.492931 28927 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:10:25.497100 28927 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:10:25.497185 28927 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:10:25.497206 28927 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:10:25.497224 28927 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:10:25.497239 28927 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:10:25.503592 28927 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:43239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785971406251336-25971-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785971406251336-25971-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.92.254:33719
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab
--principal=kudu/127.25.92.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.254:33719
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.92.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28927
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785971406251336-25971-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:10:25.504748 28927 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:10:25.505837 28927 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:10:25.515048 28932 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:25.515061 28933 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:25.515060 28935 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:10:25.515247 28927 server_base.cc:1034] running on GCE node
I20260805 23:10:25.516081 28927 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:10:25.517316 28927 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:10:25.518549 28927 hybrid_clock.cc:648] HybridClock initialized: now 1785971425518504 us; error 74 us; skew 500 ppm
Aug 05 23:10:25 dist-test-slave-ss4n krb5kdc[28914](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971425, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:10:25.521696 28927 init.cc:377] Logged in from keytab as kudu/127.25.92.254@KRBTEST.COM (short username kudu)
I20260805 23:10:25.522713 28927 webserver.cc:460] Webserver started at http://127.25.92.254:41775/ using document root <none> and password file <none>
I20260805 23:10:25.523268 28927 fs_manager.cc:359] Metadata directory not provided
I20260805 23:10:25.523329 28927 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:10:25.523495 28927 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:10:25.525693 28927 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785971406251336-25971-0/minicluster-data/master-0/data/instance:
uuid: "fc3b0a697ed74017949961b2dc30baea"
format_stamp: "Formatted at 2026-08-05 23:10:25 on dist-test-slave-ss4n"
server_key: "374e76aace68e82f10edf13b50762c72"
server_key_iv: "24e679fb0abb1dedd4963fe6832b03b6"
server_key_version: "encryptionkey@0"
I20260805 23:10:25.526235 28927 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785971406251336-25971-0/minicluster-data/master-0/wal/instance:
uuid: "fc3b0a697ed74017949961b2dc30baea"
format_stamp: "Formatted at 2026-08-05 23:10:25 on dist-test-slave-ss4n"
server_key: "374e76aace68e82f10edf13b50762c72"
server_key_iv: "24e679fb0abb1dedd4963fe6832b03b6"
server_key_version: "encryptionkey@0"
I20260805 23:10:25.530222 28927 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260805 23:10:25.532979 28942 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:25.534533 28927 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260805 23:10:25.534737 28927 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785971406251336-25971-0/minicluster-data/master-0/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785971406251336-25971-0/minicluster-data/master-0/wal
uuid: "fc3b0a697ed74017949961b2dc30baea"
format_stamp: "Formatted at 2026-08-05 23:10:25 on dist-test-slave-ss4n"
server_key: "374e76aace68e82f10edf13b50762c72"
server_key_iv: "24e679fb0abb1dedd4963fe6832b03b6"
server_key_version: "encryptionkey@0"
I20260805 23:10:25.534861 28927 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785971406251336-25971-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785971406251336-25971-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785971406251336-25971-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:10:25.562300 28927 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:10:25.566239 28927 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:10:25.566507 28927 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:10:25.574601 28927 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.254:33719
I20260805 23:10:25.575045 28995 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.254:33719 every 8 connection(s)
I20260805 23:10:25.575862 28927 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
I20260805 23:10:25.578922 28996 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:10:25.581777 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 28927
I20260805 23:10:25.581916 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785971406251336-25971-0/minicluster-data/master-0/wal/instance
I20260805 23:10:25.582231 25971 external_mini_cluster.cc:1428] Setting key 1d645c80e442c2053ac7db117a5c0658
I20260805 23:10:25.585294 28996 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc3b0a697ed74017949961b2dc30baea: Bootstrap starting.
I20260805 23:10:25.588027 28996 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fc3b0a697ed74017949961b2dc30baea: Neither blocks nor log segments found. Creating new log.
I20260805 23:10:25.588974 28996 log.cc:824] T 00000000000000000000000000000000 P fc3b0a697ed74017949961b2dc30baea: Log is configured to *not* fsync() on all Append() calls
I20260805 23:10:25.591161 28996 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc3b0a697ed74017949961b2dc30baea: No bootstrap required, opened a new log
Aug 05 23:10:25 dist-test-slave-ss4n krb5kdc[28914](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971425, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM
I20260805 23:10:25.595500 28996 raft_consensus.cc:348] T 00000000000000000000000000000000 P fc3b0a697ed74017949961b2dc30baea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc3b0a697ed74017949961b2dc30baea" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 33719 } }
I20260805 23:10:25.595785 28996 raft_consensus.cc:374] T 00000000000000000000000000000000 P fc3b0a697ed74017949961b2dc30baea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:10:25.595881 28996 raft_consensus.cc:729] T 00000000000000000000000000000000 P fc3b0a697ed74017949961b2dc30baea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc3b0a697ed74017949961b2dc30baea, State: Initialized, Role: FOLLOWER
I20260805 23:10:25.596441 28996 consensus_queue.cc:260] T 00000000000000000000000000000000 P fc3b0a697ed74017949961b2dc30baea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc3b0a697ed74017949961b2dc30baea" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 33719 } }
I20260805 23:10:25.596627 28996 raft_consensus.cc:388] T 00000000000000000000000000000000 P fc3b0a697ed74017949961b2dc30baea [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:10:25.596710 28996 raft_consensus.cc:482] T 00000000000000000000000000000000 P fc3b0a697ed74017949961b2dc30baea [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:10:25.596834 28996 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fc3b0a697ed74017949961b2dc30baea [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:10:25.598233 28996 raft_consensus.cc:504] T 00000000000000000000000000000000 P fc3b0a697ed74017949961b2dc30baea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc3b0a697ed74017949961b2dc30baea" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 33719 } }
I20260805 23:10:25.598621 28996 leader_election.cc:302] T 00000000000000000000000000000000 P fc3b0a697ed74017949961b2dc30baea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fc3b0a697ed74017949961b2dc30baea; no voters:
I20260805 23:10:25.598985 28996 leader_election.cc:288] T 00000000000000000000000000000000 P fc3b0a697ed74017949961b2dc30baea [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 23:10:25.599206 29001 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fc3b0a697ed74017949961b2dc30baea [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:10:25.599382 28999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:25.583669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40314 (local address 127.25.92.254:33719)
0805 23:10:25.584180 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:25.584208 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:25.584248 (+ 40us) server_negotiation.cc:407] Connection header received
0805 23:10:25.584924 (+ 676us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:25.584943 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:25.585294 (+ 351us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:25.585687 (+ 393us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:25.586622 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:25.587684 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:25.588934 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:25.589659 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:25.589884 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:25.593033 (+ 3149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:25.593041 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:25.593061 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:25.593094 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:25.595458 (+ 2364us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:25.596065 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:25.596071 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:25.596077 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:25.596242 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:25.596857 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:25.596861 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:25.596863 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:25.597441 (+ 578us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:25.597743 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:25.597958 (+ 215us) server_negotiation.cc:299] Negotiation successful
0805 23:10:25.598304 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":161,"threads_started":1}
I20260805 23:10:25.599465 29001 raft_consensus.cc:686] T 00000000000000000000000000000000 P fc3b0a697ed74017949961b2dc30baea [term 1 LEADER]: Becoming Leader. State: Replica: fc3b0a697ed74017949961b2dc30baea, State: Running, Role: LEADER
I20260805 23:10:25.600065 29001 consensus_queue.cc:237] T 00000000000000000000000000000000 P fc3b0a697ed74017949961b2dc30baea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc3b0a697ed74017949961b2dc30baea" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 33719 } }
I20260805 23:10:25.600435 28996 sys_catalog.cc:564] T 00000000000000000000000000000000 P fc3b0a697ed74017949961b2dc30baea [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:10:25.602016 29003 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc3b0a697ed74017949961b2dc30baea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fc3b0a697ed74017949961b2dc30baea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc3b0a697ed74017949961b2dc30baea" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 33719 } } }
I20260805 23:10:25.602202 29003 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc3b0a697ed74017949961b2dc30baea [sys.catalog]: This master's current role is: LEADER
I20260805 23:10:25.602646 29013 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:10:25.602651 29002 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc3b0a697ed74017949961b2dc30baea [sys.catalog]: SysCatalogTable state changed. Reason: New leader fc3b0a697ed74017949961b2dc30baea. Latest consensus state: current_term: 1 leader_uuid: "fc3b0a697ed74017949961b2dc30baea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc3b0a697ed74017949961b2dc30baea" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 33719 } } }
I20260805 23:10:25.602835 29002 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc3b0a697ed74017949961b2dc30baea [sys.catalog]: This master's current role is: LEADER
I20260805 23:10:25.603014 25971 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260805 23:10:25.606098 29013 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:10:25.612499 29013 catalog_manager.cc:1347] Generated new cluster ID: 3f928591418846c2aae935c169ccf0da
I20260805 23:10:25.612709 29013 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:10:25.625360 28999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:25.619619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40328 (local address 127.25.92.254:33719)
0805 23:10:25.619794 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:25.619800 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:25.620092 (+ 292us) server_negotiation.cc:407] Connection header received
0805 23:10:25.620321 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:25.620324 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:25.620399 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:25.620510 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:25.621190 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:25.621878 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:25.622847 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:25.623136 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:25.623267 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:25.624264 (+ 997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:25.624269 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:25.624279 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:10:25.624787 (+ 508us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:25.624819 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:10:25.624837 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:25.624954 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:25.625089 (+ 135us) server_negotiation.cc:299] Negotiation successful
0805 23:10:25.625159 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":85}
I20260805 23:10:25.636287 29013 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:10:25.637751 29013 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:10:25.646857 29013 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fc3b0a697ed74017949961b2dc30baea: Generated new TSK 0
I20260805 23:10:25.647816 29013 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260805 23:10:25.656157 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 28927
2026-08-05T23:10:25Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1527 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971406251336-25971-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 23:10:25 dist-test-slave-ss4n krb5kdc[29029](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 23:10:25 dist-test-slave-ss4n krb5kdc[29029](info): set up 2 sockets
Aug 05 23:10:25 dist-test-slave-ss4n krb5kdc[29029](info): commencing operation
krb5kdc: starting...
W20260805 23:10:26.759691 25971 mini_kdc.cc:121] Time spent starting KDC: real 1.081s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971406251336-25971-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971406251336-25971-0/krb5kdc/test-user.keytab.
Aug 05 23:10:26 dist-test-slave-ss4n krb5kdc[29029](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971426, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T23:10:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:10:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.92.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.92.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.92.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab.
Entry for principal kudu/127.25.92.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab.
I20260805 23:10:26.893265 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971406251336-25971-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971406251336-25971-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971406251336-25971-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.254:45543
--webserver_interface=127.25.92.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab
--principal=kudu/127.25.92.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:37737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.92.254:45543
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:10:27.019757 29042 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:10:27.020057 29042 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:10:27.020103 29042 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:10:27.024330 29042 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:10:27.024396 29042 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:10:27.024415 29042 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:10:27.024432 29042 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:10:27.024447 29042 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:10:27.029863 29042 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:37737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971406251336-25971-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971406251336-25971-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.92.254:45543
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab
--principal=kudu/127.25.92.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.254:45543
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.92.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29042
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971406251336-25971-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:10:27.030890 29042 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:10:27.031909 29042 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:10:27.040660 29050 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:27.040628 29048 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:27.040644 29047 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:10:27.041056 29042 server_base.cc:1034] running on GCE node
I20260805 23:10:27.041689 29042 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:10:27.042853 29042 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:10:27.044051 29042 hybrid_clock.cc:648] HybridClock initialized: now 1785971427044038 us; error 30 us; skew 500 ppm
Aug 05 23:10:27 dist-test-slave-ss4n krb5kdc[29029](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971427, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:10:27.047230 29042 init.cc:377] Logged in from keytab as kudu/127.25.92.254@KRBTEST.COM (short username kudu)
I20260805 23:10:27.048218 29042 webserver.cc:460] Webserver started at http://127.25.92.254:43237/ using document root <none> and password file <none>
I20260805 23:10:27.048857 29042 fs_manager.cc:359] Metadata directory not provided
I20260805 23:10:27.048957 29042 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:10:27.049199 29042 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:10:27.051360 29042 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971406251336-25971-0/minicluster-data/master-0/data/instance:
uuid: "6b3dcbdcdfb04987891a16787e3f6a47"
format_stamp: "Formatted at 2026-08-05 23:10:27 on dist-test-slave-ss4n"
server_key: "6fb55a10a3a19452d9313bd995bea4e0"
server_key_iv: "1f7c80efaed768a7706fc4c3b22d43e3"
server_key_version: "encryptionkey@0"
I20260805 23:10:27.051942 29042 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971406251336-25971-0/minicluster-data/master-0/wal/instance:
uuid: "6b3dcbdcdfb04987891a16787e3f6a47"
format_stamp: "Formatted at 2026-08-05 23:10:27 on dist-test-slave-ss4n"
server_key: "6fb55a10a3a19452d9313bd995bea4e0"
server_key_iv: "1f7c80efaed768a7706fc4c3b22d43e3"
server_key_version: "encryptionkey@0"
I20260805 23:10:27.055903 29042 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260805 23:10:27.058498 29057 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:27.059785 29042 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260805 23:10:27.059944 29042 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971406251336-25971-0/minicluster-data/master-0/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971406251336-25971-0/minicluster-data/master-0/wal
uuid: "6b3dcbdcdfb04987891a16787e3f6a47"
format_stamp: "Formatted at 2026-08-05 23:10:27 on dist-test-slave-ss4n"
server_key: "6fb55a10a3a19452d9313bd995bea4e0"
server_key_iv: "1f7c80efaed768a7706fc4c3b22d43e3"
server_key_version: "encryptionkey@0"
I20260805 23:10:27.060070 29042 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971406251336-25971-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971406251336-25971-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971406251336-25971-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:10:27.090062 29042 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:10:27.093621 29042 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:10:27.093842 29042 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:10:27.101289 29042 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.254:45543
I20260805 23:10:27.101701 29109 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.254:45543 every 8 connection(s)
I20260805 23:10:27.102393 29042 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
I20260805 23:10:27.105347 29110 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:10:27.110911 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 29042
I20260805 23:10:27.110960 29110 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b3dcbdcdfb04987891a16787e3f6a47: Bootstrap starting.
I20260805 23:10:27.111056 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971406251336-25971-0/minicluster-data/master-0/wal/instance
I20260805 23:10:27.111342 25971 external_mini_cluster.cc:1428] Setting key 459f703a898bbe78f31b11f3bf948eca
I20260805 23:10:27.113689 29110 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6b3dcbdcdfb04987891a16787e3f6a47: Neither blocks nor log segments found. Creating new log.
I20260805 23:10:27.114418 29110 log.cc:824] T 00000000000000000000000000000000 P 6b3dcbdcdfb04987891a16787e3f6a47: Log is configured to *not* fsync() on all Append() calls
I20260805 23:10:27.116518 29110 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b3dcbdcdfb04987891a16787e3f6a47: No bootstrap required, opened a new log
Aug 05 23:10:27 dist-test-slave-ss4n krb5kdc[29029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971426, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM
I20260805 23:10:27.119984 29110 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6b3dcbdcdfb04987891a16787e3f6a47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b3dcbdcdfb04987891a16787e3f6a47" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 45543 } }
I20260805 23:10:27.120297 29110 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6b3dcbdcdfb04987891a16787e3f6a47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:10:27.120375 29110 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6b3dcbdcdfb04987891a16787e3f6a47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b3dcbdcdfb04987891a16787e3f6a47, State: Initialized, Role: FOLLOWER
I20260805 23:10:27.120893 29110 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6b3dcbdcdfb04987891a16787e3f6a47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b3dcbdcdfb04987891a16787e3f6a47" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 45543 } }
I20260805 23:10:27.121076 29110 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6b3dcbdcdfb04987891a16787e3f6a47 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:10:27.121161 29110 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6b3dcbdcdfb04987891a16787e3f6a47 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:10:27.121294 29110 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6b3dcbdcdfb04987891a16787e3f6a47 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:10:27.122671 29110 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6b3dcbdcdfb04987891a16787e3f6a47 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b3dcbdcdfb04987891a16787e3f6a47" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 45543 } }
I20260805 23:10:27.123060 29110 leader_election.cc:302] T 00000000000000000000000000000000 P 6b3dcbdcdfb04987891a16787e3f6a47 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6b3dcbdcdfb04987891a16787e3f6a47; no voters:
I20260805 23:10:27.123445 29110 leader_election.cc:288] T 00000000000000000000000000000000 P 6b3dcbdcdfb04987891a16787e3f6a47 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 23:10:27.123579 29115 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6b3dcbdcdfb04987891a16787e3f6a47 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:10:27.123795 29115 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6b3dcbdcdfb04987891a16787e3f6a47 [term 1 LEADER]: Becoming Leader. State: Replica: 6b3dcbdcdfb04987891a16787e3f6a47, State: Running, Role: LEADER
I20260805 23:10:27.124205 29115 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6b3dcbdcdfb04987891a16787e3f6a47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b3dcbdcdfb04987891a16787e3f6a47" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 45543 } }
I20260805 23:10:27.124845 29110 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6b3dcbdcdfb04987891a16787e3f6a47 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:10:27.126219 29117 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b3dcbdcdfb04987891a16787e3f6a47 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6b3dcbdcdfb04987891a16787e3f6a47. Latest consensus state: current_term: 1 leader_uuid: "6b3dcbdcdfb04987891a16787e3f6a47" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b3dcbdcdfb04987891a16787e3f6a47" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 45543 } } }
I20260805 23:10:27.126350 29117 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b3dcbdcdfb04987891a16787e3f6a47 [sys.catalog]: This master's current role is: LEADER
I20260805 23:10:27.126549 29113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:27.112773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48522 (local address 127.25.92.254:45543)
0805 23:10:27.113226 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:27.113243 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:27.113277 (+ 34us) server_negotiation.cc:407] Connection header received
0805 23:10:27.114035 (+ 758us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:27.114055 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:27.114390 (+ 335us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:27.114722 (+ 332us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:27.115551 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:27.116539 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:27.117385 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:27.117929 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:27.118118 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:27.120793 (+ 2675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:27.120799 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:27.120816 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:27.120848 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:27.122981 (+ 2133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:27.123446 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:27.123452 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:27.123458 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:27.123596 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:27.123986 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:27.123990 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:27.123991 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:27.124516 (+ 525us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:27.124783 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:27.125137 (+ 354us) server_negotiation.cc:299] Negotiation successful
0805 23:10:27.125463 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":157,"threads_started":1}
I20260805 23:10:27.126680 29116 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b3dcbdcdfb04987891a16787e3f6a47 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6b3dcbdcdfb04987891a16787e3f6a47" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b3dcbdcdfb04987891a16787e3f6a47" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 45543 } } }
I20260805 23:10:27.126760 29116 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b3dcbdcdfb04987891a16787e3f6a47 [sys.catalog]: This master's current role is: LEADER
I20260805 23:10:27.126771 29125 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:10:27.129981 29125 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:10:27.137264 29125 catalog_manager.cc:1347] Generated new cluster ID: 05cd7ef9c5fb4cd196747643dd825b10
I20260805 23:10:27.137362 29125 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:10:27.147647 29125 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:10:27.148902 29125 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:10:27.164316 29125 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6b3dcbdcdfb04987891a16787e3f6a47: Generated new TSK 0
I20260805 23:10:27.165213 29125 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.92.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.92.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.92.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.193.keytab.
Entry for principal kudu/127.25.92.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.193.keytab.
I20260805 23:10:27.206871 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971406251336-25971-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971406251336-25971-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971406251336-25971-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971406251336-25971-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.92.193:0
--local_ip_for_outbound_sockets=127.25.92.193
--webserver_interface=127.25.92.193
--webserver_port=0
--tserver_master_addrs=127.25.92.254:45543
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.193.keytab
--principal=kudu/127.25.92.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:37737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:10:27.347997 29137 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:10:27.348315 29137 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:10:27.348368 29137 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:10:27.352540 29137 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:10:27.352625 29137 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:10:27.352689 29137 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.92.193
I20260805 23:10:27.358289 29137 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:37737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971406251336-25971-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971406251336-25971-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.193.keytab
--principal=kudu/127.25.92.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.92.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971406251336-25971-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.92.193
--webserver_port=0
--tserver_master_addrs=127.25.92.254:45543
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29137
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.92.193
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971406251336-25971-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:10:27.359404 29137 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:10:27.360409 29137 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:10:27.370651 29143 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:27.370651 29142 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:27.370715 29145 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:10:27.370941 29137 server_base.cc:1034] running on GCE node
I20260805 23:10:27.371440 29137 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:10:27.372136 29137 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:10:27.373306 29137 hybrid_clock.cc:648] HybridClock initialized: now 1785971427373292 us; error 27 us; skew 500 ppm
Aug 05 23:10:27 dist-test-slave-ss4n krb5kdc[29029](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971427, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:10:27.376453 29137 init.cc:377] Logged in from keytab as kudu/127.25.92.193@KRBTEST.COM (short username kudu)
I20260805 23:10:27.377417 29137 webserver.cc:460] Webserver started at http://127.25.92.193:33185/ using document root <none> and password file <none>
I20260805 23:10:27.378098 29137 fs_manager.cc:359] Metadata directory not provided
I20260805 23:10:27.378165 29137 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:10:27.378410 29137 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:10:27.380573 29137 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971406251336-25971-0/minicluster-data/ts-0/data/instance:
uuid: "ca748302fe504f269f1bb765a0d68679"
format_stamp: "Formatted at 2026-08-05 23:10:27 on dist-test-slave-ss4n"
server_key: "ede6b214bbc7b4702db9bc4cf1b349b5"
server_key_iv: "cd218b09112013d22aabb420c788f15c"
server_key_version: "encryptionkey@0"
I20260805 23:10:27.381151 29137 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971406251336-25971-0/minicluster-data/ts-0/wal/instance:
uuid: "ca748302fe504f269f1bb765a0d68679"
format_stamp: "Formatted at 2026-08-05 23:10:27 on dist-test-slave-ss4n"
server_key: "ede6b214bbc7b4702db9bc4cf1b349b5"
server_key_iv: "cd218b09112013d22aabb420c788f15c"
server_key_version: "encryptionkey@0"
I20260805 23:10:27.385226 29137 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260805 23:10:27.387840 29152 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:27.389217 29137 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260805 23:10:27.389367 29137 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971406251336-25971-0/minicluster-data/ts-0/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971406251336-25971-0/minicluster-data/ts-0/wal
uuid: "ca748302fe504f269f1bb765a0d68679"
format_stamp: "Formatted at 2026-08-05 23:10:27 on dist-test-slave-ss4n"
server_key: "ede6b214bbc7b4702db9bc4cf1b349b5"
server_key_iv: "cd218b09112013d22aabb420c788f15c"
server_key_version: "encryptionkey@0"
I20260805 23:10:27.389519 29137 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971406251336-25971-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971406251336-25971-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971406251336-25971-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:10:27.402146 29137 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:10:27.405927 29137 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:10:27.406150 29137 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:10:27.406838 29137 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:10:27.407752 29137 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:10:27.407828 29137 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:27.407903 29137 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:10:27.407940 29137 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:27.417129 29137 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.193:37051
I20260805 23:10:27.417150 29265 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.193:37051 every 8 connection(s)
I20260805 23:10:27.418260 29137 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971406251336-25971-0/minicluster-data/ts-0/data/info.pb
I20260805 23:10:27.425127 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 29137
I20260805 23:10:27.425256 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971406251336-25971-0/minicluster-data/ts-0/wal/instance
I20260805 23:10:27.425813 25971 external_mini_cluster.cc:1428] Setting key c7cc983e91ed9e5a07939666db99639f
Aug 05 23:10:27 dist-test-slave-ss4n krb5kdc[29029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971427, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.193@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM
I20260805 23:10:27.434840 29113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:27.420295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.193:40473 (local address 127.25.92.254:45543)
0805 23:10:27.420483 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:27.420487 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:27.421460 (+ 973us) server_negotiation.cc:407] Connection header received
0805 23:10:27.422891 (+ 1431us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:27.422895 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:27.422954 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:27.423054 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:27.424265 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:27.424850 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:27.426423 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:27.426865 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:27.426982 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:27.429918 (+ 2936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:27.429924 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:27.429927 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:27.429970 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:27.431858 (+ 1888us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:27.432674 (+ 816us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:27.432678 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:27.432679 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:27.432767 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:27.433256 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:27.433258 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:27.433260 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:27.433548 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:27.433690 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:27.434555 (+ 865us) server_negotiation.cc:299] Negotiation successful
0805 23:10:27.434695 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260805 23:10:27.435962 29268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:27.420776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:45543 (local address 127.25.92.193:40473)
0805 23:10:27.421291 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:27.421338 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:27.422665 (+ 1327us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:27.423232 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:27.423239 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:27.423629 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:27.424108 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:27.424121 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:27.424984 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:27.424987 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:27.426270 (+ 1283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:27.426286 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:27.427062 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:27.427067 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:27.427273 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:27.428085 (+ 812us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:10:27.428101 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:27.429700 (+ 1599us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:10:27.432001 (+ 2301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:27.432011 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:27.432033 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:27.432478 (+ 445us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:27.432879 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:27.432885 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:27.432889 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:27.433103 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:27.433716 (+ 613us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:10:27.433727 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:10:27.434126 (+ 399us) client_negotiation.cc:769] Sending connection context
0805 23:10:27.434379 (+ 253us) client_negotiation.cc:240] Negotiation successful
0805 23:10:27.434799 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":155,"threads_started":1}
WARNING: no policy specified for kudu/127.25.92.194@KRBTEST.COM; defaulting to no policy
I20260805 23:10:27.437670 29266 heartbeater.cc:347] Connected to a master server at 127.25.92.254:45543
I20260805 23:10:27.437901 29266 heartbeater.cc:464] Registering TS with master...
I20260805 23:10:27.438400 29266 heartbeater.cc:511] Master 127.25.92.254:45543 requested a full tablet report, sending...
I20260805 23:10:27.440024 29074 ts_manager.cc:194] Registered new tserver with Master: ca748302fe504f269f1bb765a0d68679 (127.25.92.193:37051)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.92.194@KRBTEST.COM" created.
I20260805 23:10:27.441260 29074 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.92.193@KRBTEST.COM'} at 127.25.92.193:40473
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.92.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.194.keytab.
Entry for principal kudu/127.25.92.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.194.keytab.
I20260805 23:10:27.458217 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971406251336-25971-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971406251336-25971-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971406251336-25971-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971406251336-25971-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.92.194:0
--local_ip_for_outbound_sockets=127.25.92.194
--webserver_interface=127.25.92.194
--webserver_port=0
--tserver_master_addrs=127.25.92.254:45543
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.194.keytab
--principal=kudu/127.25.92.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:37737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:10:27.576758 29271 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:10:27.577059 29271 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:10:27.577140 29271 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:10:27.581990 29271 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:10:27.582059 29271 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:10:27.582162 29271 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.92.194
I20260805 23:10:27.588447 29271 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:37737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971406251336-25971-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971406251336-25971-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.194.keytab
--principal=kudu/127.25.92.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.92.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971406251336-25971-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.92.194
--webserver_port=0
--tserver_master_addrs=127.25.92.254:45543
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29271
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.92.194
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971406251336-25971-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:10:27.589653 29271 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:10:27.590759 29271 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:10:27.601857 29276 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:27.601894 29277 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:27.601857 29279 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:10:27.602231 29271 server_base.cc:1034] running on GCE node
I20260805 23:10:27.602710 29271 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:10:27.603433 29271 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:10:27.604727 29271 hybrid_clock.cc:648] HybridClock initialized: now 1785971427604689 us; error 52 us; skew 500 ppm
Aug 05 23:10:27 dist-test-slave-ss4n krb5kdc[29029](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971427, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:10:27.607951 29271 init.cc:377] Logged in from keytab as kudu/127.25.92.194@KRBTEST.COM (short username kudu)
I20260805 23:10:27.608975 29271 webserver.cc:460] Webserver started at http://127.25.92.194:33939/ using document root <none> and password file <none>
I20260805 23:10:27.609702 29271 fs_manager.cc:359] Metadata directory not provided
I20260805 23:10:27.609812 29271 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:10:27.610069 29271 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:10:27.612275 29271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971406251336-25971-0/minicluster-data/ts-1/data/instance:
uuid: "caa45c18a68d4189829c4b3c62cc163f"
format_stamp: "Formatted at 2026-08-05 23:10:27 on dist-test-slave-ss4n"
server_key: "e2107a6cb4c093981efa05743066ecd6"
server_key_iv: "80a80e88b716909ec3509ed1c735b780"
server_key_version: "encryptionkey@0"
I20260805 23:10:27.612932 29271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971406251336-25971-0/minicluster-data/ts-1/wal/instance:
uuid: "caa45c18a68d4189829c4b3c62cc163f"
format_stamp: "Formatted at 2026-08-05 23:10:27 on dist-test-slave-ss4n"
server_key: "e2107a6cb4c093981efa05743066ecd6"
server_key_iv: "80a80e88b716909ec3509ed1c735b780"
server_key_version: "encryptionkey@0"
I20260805 23:10:27.617112 29271 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260805 23:10:27.619848 29286 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:27.621143 29271 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260805 23:10:27.621297 29271 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971406251336-25971-0/minicluster-data/ts-1/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971406251336-25971-0/minicluster-data/ts-1/wal
uuid: "caa45c18a68d4189829c4b3c62cc163f"
format_stamp: "Formatted at 2026-08-05 23:10:27 on dist-test-slave-ss4n"
server_key: "e2107a6cb4c093981efa05743066ecd6"
server_key_iv: "80a80e88b716909ec3509ed1c735b780"
server_key_version: "encryptionkey@0"
I20260805 23:10:27.621443 29271 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971406251336-25971-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971406251336-25971-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971406251336-25971-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:10:27.662323 29271 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:10:27.666184 29271 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:10:27.666397 29271 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:10:27.667007 29271 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:10:27.667851 29271 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:10:27.667923 29271 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:27.667992 29271 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:10:27.668045 29271 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:27.677049 29271 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.194:44129
I20260805 23:10:27.677078 29399 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.194:44129 every 8 connection(s)
I20260805 23:10:27.678141 29271 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971406251336-25971-0/minicluster-data/ts-1/data/info.pb
I20260805 23:10:27.686384 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 29271
I20260805 23:10:27.686544 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971406251336-25971-0/minicluster-data/ts-1/wal/instance
I20260805 23:10:27.686844 25971 external_mini_cluster.cc:1428] Setting key c83a50469eeab9b234d02f5e1a4cc6fc
Aug 05 23:10:27 dist-test-slave-ss4n krb5kdc[29029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971427, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.194@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM
I20260805 23:10:27.694339 29113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:27.680006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.194:35631 (local address 127.25.92.254:45543)
0805 23:10:27.680203 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:27.680208 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:27.681168 (+ 960us) server_negotiation.cc:407] Connection header received
0805 23:10:27.682801 (+ 1633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:27.682804 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:27.682855 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:27.682935 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:27.684238 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:27.684865 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:27.686578 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:27.686836 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:27.687000 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:27.690081 (+ 3081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:27.690084 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:27.690086 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:27.690112 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:27.691746 (+ 1634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:27.692345 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:27.692354 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:27.692355 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:27.692445 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:27.692894 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:27.692900 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:27.692905 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:27.693172 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:27.693293 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:27.693995 (+ 702us) server_negotiation.cc:299] Negotiation successful
0805 23:10:27.694141 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260805 23:10:27.695288 29402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:27.680487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:45543 (local address 127.25.92.194:35631)
0805 23:10:27.680989 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:27.681036 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:27.682555 (+ 1519us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:27.683162 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:27.683170 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:27.683560 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:27.684045 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:27.684067 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:27.685085 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:27.685091 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:27.686392 (+ 1301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:27.686407 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:27.687064 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:27.687069 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:27.687310 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:27.688024 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:10:27.688043 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:27.689879 (+ 1836us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:10:27.691890 (+ 2011us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:27.691898 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:27.691916 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:27.692216 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:27.692557 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:27.692562 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:27.692566 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:27.692777 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:27.693304 (+ 527us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:10:27.693314 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:10:27.693706 (+ 392us) client_negotiation.cc:769] Sending connection context
0805 23:10:27.693924 (+ 218us) client_negotiation.cc:240] Negotiation successful
0805 23:10:27.694308 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":124,"threads_started":1}
I20260805 23:10:27.696904 29400 heartbeater.cc:347] Connected to a master server at 127.25.92.254:45543
I20260805 23:10:27.697183 29400 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.25.92.195@KRBTEST.COM; defaulting to no policy
I20260805 23:10:27.697813 29400 heartbeater.cc:511] Master 127.25.92.254:45543 requested a full tablet report, sending...
I20260805 23:10:27.699082 29074 ts_manager.cc:194] Registered new tserver with Master: caa45c18a68d4189829c4b3c62cc163f (127.25.92.194:44129)
I20260805 23:10:27.699759 29074 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.92.194@KRBTEST.COM'} at 127.25.92.194:35631
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.92.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.92.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.195.keytab.
Entry for principal kudu/127.25.92.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.195.keytab.
I20260805 23:10:27.717854 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971406251336-25971-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971406251336-25971-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971406251336-25971-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971406251336-25971-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.92.195:0
--local_ip_for_outbound_sockets=127.25.92.195
--webserver_interface=127.25.92.195
--webserver_port=0
--tserver_master_addrs=127.25.92.254:45543
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.195.keytab
--principal=kudu/127.25.92.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:37737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:10:27.831216 29405 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:10:27.831499 29405 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:10:27.831543 29405 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:10:27.835752 29405 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:10:27.835821 29405 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:10:27.835911 29405 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.92.195
I20260805 23:10:27.842293 29405 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:37737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971406251336-25971-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971406251336-25971-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.195.keytab
--principal=kudu/127.25.92.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.92.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971406251336-25971-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.92.195
--webserver_port=0
--tserver_master_addrs=127.25.92.254:45543
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29405
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.92.195
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971406251336-25971-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:10:27.843673 29405 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:10:27.844857 29405 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:10:27.855845 29410 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:27.855916 29413 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:27.855938 29411 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:10:27.856140 29405 server_base.cc:1034] running on GCE node
I20260805 23:10:27.856680 29405 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:10:27.857430 29405 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:10:27.858680 29405 hybrid_clock.cc:648] HybridClock initialized: now 1785971427858658 us; error 27 us; skew 500 ppm
Aug 05 23:10:27 dist-test-slave-ss4n krb5kdc[29029](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971427, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:10:27.862259 29405 init.cc:377] Logged in from keytab as kudu/127.25.92.195@KRBTEST.COM (short username kudu)
I20260805 23:10:27.863422 29405 webserver.cc:460] Webserver started at http://127.25.92.195:37713/ using document root <none> and password file <none>
I20260805 23:10:27.864161 29405 fs_manager.cc:359] Metadata directory not provided
I20260805 23:10:27.864233 29405 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:10:27.864519 29405 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:10:27.866816 29405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971406251336-25971-0/minicluster-data/ts-2/data/instance:
uuid: "98299db7b2014b938ee9703a8c5f006b"
format_stamp: "Formatted at 2026-08-05 23:10:27 on dist-test-slave-ss4n"
server_key: "7c813570589ad7cdbb6103daf4f1661b"
server_key_iv: "b8aafbb492c8a806cb56d8e5c844a507"
server_key_version: "encryptionkey@0"
I20260805 23:10:27.867434 29405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971406251336-25971-0/minicluster-data/ts-2/wal/instance:
uuid: "98299db7b2014b938ee9703a8c5f006b"
format_stamp: "Formatted at 2026-08-05 23:10:27 on dist-test-slave-ss4n"
server_key: "7c813570589ad7cdbb6103daf4f1661b"
server_key_iv: "b8aafbb492c8a806cb56d8e5c844a507"
server_key_version: "encryptionkey@0"
I20260805 23:10:27.871644 29405 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260805 23:10:27.874678 29420 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:27.876017 29405 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260805 23:10:27.876164 29405 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971406251336-25971-0/minicluster-data/ts-2/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971406251336-25971-0/minicluster-data/ts-2/wal
uuid: "98299db7b2014b938ee9703a8c5f006b"
format_stamp: "Formatted at 2026-08-05 23:10:27 on dist-test-slave-ss4n"
server_key: "7c813570589ad7cdbb6103daf4f1661b"
server_key_iv: "b8aafbb492c8a806cb56d8e5c844a507"
server_key_version: "encryptionkey@0"
I20260805 23:10:27.876291 29405 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971406251336-25971-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971406251336-25971-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971406251336-25971-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:10:27.890422 29405 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:10:27.893637 29405 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:10:27.893864 29405 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:10:27.894508 29405 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:10:27.895309 29405 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:10:27.895355 29405 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:27.895418 29405 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:10:27.895450 29405 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:27.904677 29405 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.195:34371
I20260805 23:10:27.904716 29534 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.195:34371 every 8 connection(s)
I20260805 23:10:27.905704 29405 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971406251336-25971-0/minicluster-data/ts-2/data/info.pb
I20260805 23:10:27.914877 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 29405
I20260805 23:10:27.915026 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971406251336-25971-0/minicluster-data/ts-2/wal/instance
I20260805 23:10:27.915403 25971 external_mini_cluster.cc:1428] Setting key 56ab1f5a72b0fde7914b29f0dedb4c31
Aug 05 23:10:27 dist-test-slave-ss4n krb5kdc[29029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971427, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.195@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM
I20260805 23:10:27.922760 29113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:27.907833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.195:49773 (local address 127.25.92.254:45543)
0805 23:10:27.908015 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:27.908019 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:27.908918 (+ 899us) server_negotiation.cc:407] Connection header received
0805 23:10:27.910406 (+ 1488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:27.910409 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:27.910469 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:27.910582 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:27.911795 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:27.912339 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:27.913534 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:27.913992 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:27.914096 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:27.917703 (+ 3607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:27.917710 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:27.917717 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:27.917756 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:27.919952 (+ 2196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:27.920717 (+ 765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:27.920721 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:27.920741 (+ 20us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:27.920840 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:27.921245 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:27.921250 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:27.921252 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:27.921569 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:27.921686 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:27.922402 (+ 716us) server_negotiation.cc:299] Negotiation successful
0805 23:10:27.922562 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260805 23:10:27.923717 29537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:27.908198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:45543 (local address 127.25.92.195:49773)
0805 23:10:27.908752 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:27.908796 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:27.910186 (+ 1390us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:27.910757 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:27.910764 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:27.911149 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:27.911638 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:27.911651 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:27.912489 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:27.912493 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:27.913353 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:27.913362 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:27.914149 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:27.914153 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:27.914318 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:27.915123 (+ 805us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:10:27.915150 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:27.917427 (+ 2277us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:10:27.920097 (+ 2670us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:27.920104 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:27.920124 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:27.920563 (+ 439us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:27.920947 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:27.920952 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:27.920955 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:27.921146 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:27.921685 (+ 539us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:10:27.921693 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:10:27.922056 (+ 363us) client_negotiation.cc:769] Sending connection context
0805 23:10:27.922301 (+ 245us) client_negotiation.cc:240] Negotiation successful
0805 23:10:27.922690 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":139,"threads_started":1}
I20260805 23:10:27.925196 29535 heartbeater.cc:347] Connected to a master server at 127.25.92.254:45543
I20260805 23:10:27.925539 29535 heartbeater.cc:464] Registering TS with master...
I20260805 23:10:27.926292 29535 heartbeater.cc:511] Master 127.25.92.254:45543 requested a full tablet report, sending...
I20260805 23:10:27.927582 29074 ts_manager.cc:194] Registered new tserver with Master: 98299db7b2014b938ee9703a8c5f006b (127.25.92.195:34371)
I20260805 23:10:27.928224 29074 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.92.195@KRBTEST.COM'} at 127.25.92.195:49773
I20260805 23:10:27.930702 25971 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 05 23:10:27 dist-test-slave-ss4n krb5kdc[29029](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971427, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 05 23:10:27 dist-test-slave-ss4n krb5kdc[29029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971427, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM
I20260805 23:10:27.959867 29113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:27.950602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48534 (local address 127.25.92.254:45543)
0805 23:10:27.950743 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:27.950747 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:27.950782 (+ 35us) server_negotiation.cc:407] Connection header received
0805 23:10:27.950983 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:27.950987 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:27.951035 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:27.951119 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:27.951892 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:27.952423 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:27.953389 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:27.953751 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:27.953860 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:27.956580 (+ 2720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:27.956583 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:27.956586 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:27.956611 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:27.958144 (+ 1533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:27.958621 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:27.958624 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:27.958625 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:27.958704 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:27.958957 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:27.958959 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:27.958961 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:27.959200 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:27.959301 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:27.959562 (+ 261us) server_negotiation.cc:299] Negotiation successful
0805 23:10:27.959661 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
W20260805 23:10:27.960345 29074 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:48534
I20260805 23:10:27.961393 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 29137
I20260805 23:10:27.968618 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 29271
I20260805 23:10:27.975534 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 29405
I20260805 23:10:27.982054 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 29042
2026-08-05T23:10:27Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2327 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971406251336-25971-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 23:10:28 dist-test-slave-ss4n krb5kdc[29547](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 23:10:28 dist-test-slave-ss4n krb5kdc[29547](info): set up 2 sockets
Aug 05 23:10:28 dist-test-slave-ss4n krb5kdc[29547](info): commencing operation
krb5kdc: starting...
W20260805 23:10:29.099992 25971 mini_kdc.cc:121] Time spent starting KDC: real 1.094s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971406251336-25971-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971406251336-25971-0/krb5kdc/test-user.keytab.
Aug 05 23:10:29 dist-test-slave-ss4n krb5kdc[29547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971429, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T23:10:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:10:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.92.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.92.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.92.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab.
Entry for principal kudu/127.25.92.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab.
I20260805 23:10:29.233786 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971406251336-25971-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971406251336-25971-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971406251336-25971-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.254:38007
--webserver_interface=127.25.92.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab
--principal=kudu/127.25.92.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:34037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.92.254:38007
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:10:29.355649 29560 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:10:29.356036 29560 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:10:29.356122 29560 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:10:29.360615 29560 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:10:29.360728 29560 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:10:29.360764 29560 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:10:29.360803 29560 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:10:29.360855 29560 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:10:29.366463 29560 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:34037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971406251336-25971-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971406251336-25971-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.92.254:38007
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab
--principal=kudu/127.25.92.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.254:38007
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.92.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29560
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971406251336-25971-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:10:29.367650 29560 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:10:29.368794 29560 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:10:29.377722 29568 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:29.377779 29565 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:29.377727 29566 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:10:29.378225 29560 server_base.cc:1034] running on GCE node
I20260805 23:10:29.378881 29560 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:10:29.380093 29560 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:10:29.381307 29560 hybrid_clock.cc:648] HybridClock initialized: now 1785971429381291 us; error 32 us; skew 500 ppm
Aug 05 23:10:29 dist-test-slave-ss4n krb5kdc[29547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971429, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:10:29.384487 29560 init.cc:377] Logged in from keytab as kudu/127.25.92.254@KRBTEST.COM (short username kudu)
I20260805 23:10:29.385689 29560 webserver.cc:460] Webserver started at http://127.25.92.254:43311/ using document root <none> and password file <none>
I20260805 23:10:29.386318 29560 fs_manager.cc:359] Metadata directory not provided
I20260805 23:10:29.386415 29560 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:10:29.386667 29560 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:10:29.388978 29560 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971406251336-25971-0/minicluster-data/master-0/data/instance:
uuid: "617bb098b7c7414da732abe3313fabf5"
format_stamp: "Formatted at 2026-08-05 23:10:29 on dist-test-slave-ss4n"
server_key: "57b4011353966b95015af243cb0ca8e8"
server_key_iv: "4e1b5161bc0c8b4d7dad6ae16017da24"
server_key_version: "encryptionkey@0"
I20260805 23:10:29.389756 29560 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971406251336-25971-0/minicluster-data/master-0/wal/instance:
uuid: "617bb098b7c7414da732abe3313fabf5"
format_stamp: "Formatted at 2026-08-05 23:10:29 on dist-test-slave-ss4n"
server_key: "57b4011353966b95015af243cb0ca8e8"
server_key_iv: "4e1b5161bc0c8b4d7dad6ae16017da24"
server_key_version: "encryptionkey@0"
I20260805 23:10:29.394059 29560 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260805 23:10:29.396903 29575 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:29.398316 29560 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:10:29.398488 29560 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971406251336-25971-0/minicluster-data/master-0/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971406251336-25971-0/minicluster-data/master-0/wal
uuid: "617bb098b7c7414da732abe3313fabf5"
format_stamp: "Formatted at 2026-08-05 23:10:29 on dist-test-slave-ss4n"
server_key: "57b4011353966b95015af243cb0ca8e8"
server_key_iv: "4e1b5161bc0c8b4d7dad6ae16017da24"
server_key_version: "encryptionkey@0"
I20260805 23:10:29.398627 29560 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971406251336-25971-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971406251336-25971-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971406251336-25971-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:10:29.425834 29560 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:10:29.429730 29560 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:10:29.429970 29560 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:10:29.438135 29560 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.254:38007
I20260805 23:10:29.438151 29627 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.254:38007 every 8 connection(s)
I20260805 23:10:29.439244 29560 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
I20260805 23:10:29.441444 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 29560
I20260805 23:10:29.441606 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971406251336-25971-0/minicluster-data/master-0/wal/instance
I20260805 23:10:29.441916 25971 external_mini_cluster.cc:1428] Setting key 7d9e2b3979bc41bf2b70d869e12682c2
I20260805 23:10:29.442440 29628 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:10:29.448660 29628 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 617bb098b7c7414da732abe3313fabf5: Bootstrap starting.
I20260805 23:10:29.450980 29628 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 617bb098b7c7414da732abe3313fabf5: Neither blocks nor log segments found. Creating new log.
Aug 05 23:10:29 dist-test-slave-ss4n krb5kdc[29547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971429, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM
I20260805 23:10:29.452003 29628 log.cc:824] T 00000000000000000000000000000000 P 617bb098b7c7414da732abe3313fabf5: Log is configured to *not* fsync() on all Append() calls
I20260805 23:10:29.454041 29628 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 617bb098b7c7414da732abe3313fabf5: No bootstrap required, opened a new log
I20260805 23:10:29.457163 29631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:29.443446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51414 (local address 127.25.92.254:38007)
0805 23:10:29.443906 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:29.443922 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:29.443961 (+ 39us) server_negotiation.cc:407] Connection header received
0805 23:10:29.444668 (+ 707us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:29.444684 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:29.445017 (+ 333us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:29.445364 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:29.446284 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:29.447366 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:29.448295 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:29.448847 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:29.449009 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:29.452113 (+ 3104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:29.452123 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:29.452153 (+ 30us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:29.452192 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:29.454194 (+ 2002us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:29.454679 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:29.454691 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:29.454696 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:29.454810 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:29.455110 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:29.455115 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:29.455117 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:29.455542 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:29.455751 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:29.456083 (+ 332us) server_negotiation.cc:299] Negotiation successful
0805 23:10:29.456397 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":163,"threads_started":1}
I20260805 23:10:29.457696 29628 raft_consensus.cc:348] T 00000000000000000000000000000000 P 617bb098b7c7414da732abe3313fabf5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "617bb098b7c7414da732abe3313fabf5" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 38007 } }
I20260805 23:10:29.457939 29628 raft_consensus.cc:374] T 00000000000000000000000000000000 P 617bb098b7c7414da732abe3313fabf5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:10:29.458022 29628 raft_consensus.cc:729] T 00000000000000000000000000000000 P 617bb098b7c7414da732abe3313fabf5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 617bb098b7c7414da732abe3313fabf5, State: Initialized, Role: FOLLOWER
I20260805 23:10:29.458429 29628 consensus_queue.cc:260] T 00000000000000000000000000000000 P 617bb098b7c7414da732abe3313fabf5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "617bb098b7c7414da732abe3313fabf5" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 38007 } }
I20260805 23:10:29.458577 29628 raft_consensus.cc:388] T 00000000000000000000000000000000 P 617bb098b7c7414da732abe3313fabf5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:10:29.458662 29628 raft_consensus.cc:482] T 00000000000000000000000000000000 P 617bb098b7c7414da732abe3313fabf5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:10:29.458775 29628 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 617bb098b7c7414da732abe3313fabf5 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:10:29.459813 29628 raft_consensus.cc:504] T 00000000000000000000000000000000 P 617bb098b7c7414da732abe3313fabf5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "617bb098b7c7414da732abe3313fabf5" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 38007 } }
I20260805 23:10:29.460144 29628 leader_election.cc:302] T 00000000000000000000000000000000 P 617bb098b7c7414da732abe3313fabf5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 617bb098b7c7414da732abe3313fabf5; no voters:
I20260805 23:10:29.460448 29628 leader_election.cc:288] T 00000000000000000000000000000000 P 617bb098b7c7414da732abe3313fabf5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 23:10:29.460583 29633 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 617bb098b7c7414da732abe3313fabf5 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:10:29.460845 29633 raft_consensus.cc:686] T 00000000000000000000000000000000 P 617bb098b7c7414da732abe3313fabf5 [term 1 LEADER]: Becoming Leader. State: Replica: 617bb098b7c7414da732abe3313fabf5, State: Running, Role: LEADER
I20260805 23:10:29.461153 29633 consensus_queue.cc:237] T 00000000000000000000000000000000 P 617bb098b7c7414da732abe3313fabf5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "617bb098b7c7414da732abe3313fabf5" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 38007 } }
I20260805 23:10:29.461737 29628 sys_catalog.cc:564] T 00000000000000000000000000000000 P 617bb098b7c7414da732abe3313fabf5 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:10:29.463090 29635 sys_catalog.cc:455] T 00000000000000000000000000000000 P 617bb098b7c7414da732abe3313fabf5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 617bb098b7c7414da732abe3313fabf5. Latest consensus state: current_term: 1 leader_uuid: "617bb098b7c7414da732abe3313fabf5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "617bb098b7c7414da732abe3313fabf5" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 38007 } } }
I20260805 23:10:29.463073 29634 sys_catalog.cc:455] T 00000000000000000000000000000000 P 617bb098b7c7414da732abe3313fabf5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "617bb098b7c7414da732abe3313fabf5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "617bb098b7c7414da732abe3313fabf5" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 38007 } } }
I20260805 23:10:29.463232 29634 sys_catalog.cc:458] T 00000000000000000000000000000000 P 617bb098b7c7414da732abe3313fabf5 [sys.catalog]: This master's current role is: LEADER
I20260805 23:10:29.463232 29635 sys_catalog.cc:458] T 00000000000000000000000000000000 P 617bb098b7c7414da732abe3313fabf5 [sys.catalog]: This master's current role is: LEADER
W20260805 23:10:29.467258 29648 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 617bb098b7c7414da732abe3313fabf5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260805 23:10:29.467329 29648 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260805 23:10:29.467403 29642 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:10:29.468107 29642 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:10:29.473734 29642 catalog_manager.cc:1347] Generated new cluster ID: a52d27d6b0be47aab25fa62a0d9d41f1
I20260805 23:10:29.473834 29642 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:10:29.500353 29642 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:10:29.501425 29642 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:10:29.507519 29642 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 617bb098b7c7414da732abe3313fabf5: Generated new TSK 0
I20260805 23:10:29.508286 29642 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.92.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.92.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.92.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.193.keytab.
Entry for principal kudu/127.25.92.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.193.keytab.
I20260805 23:10:29.548846 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971406251336-25971-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971406251336-25971-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971406251336-25971-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971406251336-25971-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.92.193:0
--local_ip_for_outbound_sockets=127.25.92.193
--webserver_interface=127.25.92.193
--webserver_port=0
--tserver_master_addrs=127.25.92.254:38007
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.193.keytab
--principal=kudu/127.25.92.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:34037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:10:29.669965 29654 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:10:29.670265 29654 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:10:29.670315 29654 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:10:29.674484 29654 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:10:29.674568 29654 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:10:29.674634 29654 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.92.193
I20260805 23:10:29.680217 29654 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:34037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971406251336-25971-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971406251336-25971-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.193.keytab
--principal=kudu/127.25.92.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.92.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971406251336-25971-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.92.193
--webserver_port=0
--tserver_master_addrs=127.25.92.254:38007
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29654
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.92.193
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971406251336-25971-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:10:29.681394 29654 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:10:29.682710 29654 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:10:29.693164 29661 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:29.693173 29660 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:10:29.693328 29654 server_base.cc:1034] running on GCE node
W20260805 23:10:29.693169 29663 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:10:29.694016 29654 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:10:29.694715 29654 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:10:29.695904 29654 hybrid_clock.cc:648] HybridClock initialized: now 1785971429695867 us; error 51 us; skew 500 ppm
Aug 05 23:10:29 dist-test-slave-ss4n krb5kdc[29547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971429, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:10:29.699072 29654 init.cc:377] Logged in from keytab as kudu/127.25.92.193@KRBTEST.COM (short username kudu)
I20260805 23:10:29.700032 29654 webserver.cc:460] Webserver started at http://127.25.92.193:36775/ using document root <none> and password file <none>
I20260805 23:10:29.700608 29654 fs_manager.cc:359] Metadata directory not provided
I20260805 23:10:29.700709 29654 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:10:29.700937 29654 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:10:29.703217 29654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971406251336-25971-0/minicluster-data/ts-0/data/instance:
uuid: "b721c88ea08c4d45b39f7fee4801d732"
format_stamp: "Formatted at 2026-08-05 23:10:29 on dist-test-slave-ss4n"
server_key: "0b35062595043e8996162e6b5bc5c91d"
server_key_iv: "dfaddff4ced43c7eb4ae2e454d5e3b14"
server_key_version: "encryptionkey@0"
I20260805 23:10:29.703841 29654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971406251336-25971-0/minicluster-data/ts-0/wal/instance:
uuid: "b721c88ea08c4d45b39f7fee4801d732"
format_stamp: "Formatted at 2026-08-05 23:10:29 on dist-test-slave-ss4n"
server_key: "0b35062595043e8996162e6b5bc5c91d"
server_key_iv: "dfaddff4ced43c7eb4ae2e454d5e3b14"
server_key_version: "encryptionkey@0"
I20260805 23:10:29.707854 29654 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.005s
I20260805 23:10:29.710542 29671 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:29.711766 29654 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260805 23:10:29.711905 29654 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971406251336-25971-0/minicluster-data/ts-0/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971406251336-25971-0/minicluster-data/ts-0/wal
uuid: "b721c88ea08c4d45b39f7fee4801d732"
format_stamp: "Formatted at 2026-08-05 23:10:29 on dist-test-slave-ss4n"
server_key: "0b35062595043e8996162e6b5bc5c91d"
server_key_iv: "dfaddff4ced43c7eb4ae2e454d5e3b14"
server_key_version: "encryptionkey@0"
I20260805 23:10:29.712023 29654 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971406251336-25971-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971406251336-25971-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971406251336-25971-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:10:29.756430 29654 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:10:29.759545 29654 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:10:29.759774 29654 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:10:29.760388 29654 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:10:29.761260 29654 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:10:29.761330 29654 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:29.761404 29654 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:10:29.761452 29654 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:29.771107 29654 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.193:36487
I20260805 23:10:29.771128 29786 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.193:36487 every 8 connection(s)
I20260805 23:10:29.772068 29654 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971406251336-25971-0/minicluster-data/ts-0/data/info.pb
I20260805 23:10:29.776860 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 29654
I20260805 23:10:29.777058 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971406251336-25971-0/minicluster-data/ts-0/wal/instance
I20260805 23:10:29.778596 25971 external_mini_cluster.cc:1428] Setting key 211f2c0fbf2e14a3bc3c044171efe337
Aug 05 23:10:29 dist-test-slave-ss4n krb5kdc[29547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971429, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.193@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.25.92.194@KRBTEST.COM; defaulting to no policy
I20260805 23:10:29.792130 29631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:29.774455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.193:33067 (local address 127.25.92.254:38007)
0805 23:10:29.774616 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:29.774638 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:29.775769 (+ 1131us) server_negotiation.cc:407] Connection header received
0805 23:10:29.778097 (+ 2328us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:29.778125 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:29.778201 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:29.778308 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:29.780026 (+ 1718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:29.781000 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:29.782862 (+ 1862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:29.783138 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:29.783263 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:29.786857 (+ 3594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:29.786864 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:29.786869 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:29.786903 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:29.789025 (+ 2122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:29.789822 (+ 797us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:29.789826 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:29.789832 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:29.789954 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:29.790482 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:29.790490 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:29.790495 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:29.790786 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:29.790881 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:29.791722 (+ 841us) server_negotiation.cc:299] Negotiation successful
0805 23:10:29.791895 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260805 23:10:29.793113 29789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:29.774912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:38007 (local address 127.25.92.193:33067)
0805 23:10:29.775559 (+ 647us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:29.775622 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:29.777822 (+ 2200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:29.778547 (+ 725us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:29.778558 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:29.779123 (+ 565us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:29.779822 (+ 699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:29.779841 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:29.781151 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:29.781158 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:29.782705 (+ 1547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:29.782716 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:29.783309 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:29.783314 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:29.783589 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:29.784343 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:10:29.784372 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:29.786637 (+ 2265us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:10:29.789170 (+ 2533us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:29.789179 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:29.789202 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:29.789670 (+ 468us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:29.790101 (+ 431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:29.790104 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:29.790107 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:29.790333 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:29.790968 (+ 635us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:10:29.790979 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:10:29.791381 (+ 402us) client_negotiation.cc:769] Sending connection context
0805 23:10:29.791596 (+ 215us) client_negotiation.cc:240] Negotiation successful
0805 23:10:29.791893 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":159,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.92.194@KRBTEST.COM" created.
I20260805 23:10:29.794785 29787 heartbeater.cc:347] Connected to a master server at 127.25.92.254:38007
I20260805 23:10:29.795048 29787 heartbeater.cc:464] Registering TS with master...
I20260805 23:10:29.795578 29787 heartbeater.cc:511] Master 127.25.92.254:38007 requested a full tablet report, sending...
I20260805 23:10:29.797394 29592 ts_manager.cc:194] Registered new tserver with Master: b721c88ea08c4d45b39f7fee4801d732 (127.25.92.193:36487)
I20260805 23:10:29.799106 29592 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.92.193@KRBTEST.COM'} at 127.25.92.193:33067
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.92.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.194.keytab.
Entry for principal kudu/127.25.92.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.194.keytab.
I20260805 23:10:29.813592 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971406251336-25971-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971406251336-25971-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971406251336-25971-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971406251336-25971-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.92.194:0
--local_ip_for_outbound_sockets=127.25.92.194
--webserver_interface=127.25.92.194
--webserver_port=0
--tserver_master_addrs=127.25.92.254:38007
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.194.keytab
--principal=kudu/127.25.92.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:34037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:10:29.934000 29792 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:10:29.934283 29792 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:10:29.934326 29792 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:10:29.938490 29792 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:10:29.938561 29792 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:10:29.938623 29792 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.92.194
I20260805 23:10:29.944150 29792 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:34037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971406251336-25971-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971406251336-25971-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.194.keytab
--principal=kudu/127.25.92.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.92.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971406251336-25971-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.92.194
--webserver_port=0
--tserver_master_addrs=127.25.92.254:38007
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29792
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.92.194
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971406251336-25971-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:10:29.945207 29792 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:10:29.946242 29792 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:10:29.956172 29797 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:10:29.956346 29792 server_base.cc:1034] running on GCE node
W20260805 23:10:29.956164 29798 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:29.956184 29800 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:10:29.957101 29792 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:10:29.957804 29792 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:10:29.958973 29792 hybrid_clock.cc:648] HybridClock initialized: now 1785971429958953 us; error 34 us; skew 500 ppm
Aug 05 23:10:29 dist-test-slave-ss4n krb5kdc[29547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971429, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:10:29.966302 29792 init.cc:377] Logged in from keytab as kudu/127.25.92.194@KRBTEST.COM (short username kudu)
I20260805 23:10:29.967383 29792 webserver.cc:460] Webserver started at http://127.25.92.194:45865/ using document root <none> and password file <none>
I20260805 23:10:29.967970 29792 fs_manager.cc:359] Metadata directory not provided
I20260805 23:10:29.968034 29792 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:10:29.968259 29792 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:10:29.970466 29792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971406251336-25971-0/minicluster-data/ts-1/data/instance:
uuid: "b94436eb1acc4f648a487633a453e7d1"
format_stamp: "Formatted at 2026-08-05 23:10:29 on dist-test-slave-ss4n"
server_key: "d54bc072665df8e20d5e7d9ecf11d016"
server_key_iv: "eb9cde50550a10b8a770ae7ef52fe92f"
server_key_version: "encryptionkey@0"
I20260805 23:10:29.971036 29792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971406251336-25971-0/minicluster-data/ts-1/wal/instance:
uuid: "b94436eb1acc4f648a487633a453e7d1"
format_stamp: "Formatted at 2026-08-05 23:10:29 on dist-test-slave-ss4n"
server_key: "d54bc072665df8e20d5e7d9ecf11d016"
server_key_iv: "eb9cde50550a10b8a770ae7ef52fe92f"
server_key_version: "encryptionkey@0"
I20260805 23:10:29.974850 29792 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260805 23:10:29.977363 29807 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:29.978721 29792 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260805 23:10:29.978852 29792 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971406251336-25971-0/minicluster-data/ts-1/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971406251336-25971-0/minicluster-data/ts-1/wal
uuid: "b94436eb1acc4f648a487633a453e7d1"
format_stamp: "Formatted at 2026-08-05 23:10:29 on dist-test-slave-ss4n"
server_key: "d54bc072665df8e20d5e7d9ecf11d016"
server_key_iv: "eb9cde50550a10b8a770ae7ef52fe92f"
server_key_version: "encryptionkey@0"
I20260805 23:10:29.978955 29792 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971406251336-25971-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971406251336-25971-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971406251336-25971-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:10:29.991715 29792 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:10:29.994834 29792 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:10:29.995045 29792 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:10:29.995659 29792 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:10:29.996500 29792 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:10:29.996568 29792 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:29.996634 29792 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:10:29.996698 29792 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:30.006048 29792 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.194:39103
I20260805 23:10:30.006086 29921 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.194:39103 every 8 connection(s)
I20260805 23:10:30.006976 29792 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971406251336-25971-0/minicluster-data/ts-1/data/info.pb
I20260805 23:10:30.010634 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 29792
I20260805 23:10:30.010772 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971406251336-25971-0/minicluster-data/ts-1/wal/instance
I20260805 23:10:30.011032 25971 external_mini_cluster.cc:1428] Setting key ff61ea584c77d2c8277457b4e53bfa3c
Aug 05 23:10:30 dist-test-slave-ss4n krb5kdc[29547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971429, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.194@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.25.92.195@KRBTEST.COM; defaulting to no policy
I20260805 23:10:30.022817 29631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:30.008827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.194:58589 (local address 127.25.92.254:38007)
0805 23:10:30.008973 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:30.008977 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:30.009894 (+ 917us) server_negotiation.cc:407] Connection header received
0805 23:10:30.011377 (+ 1483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:30.011382 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:30.011443 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:30.011519 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:30.013009 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:30.013893 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:30.015086 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:30.015317 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:30.015427 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:30.018473 (+ 3046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:30.018480 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:30.018485 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:30.018523 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:30.020505 (+ 1982us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:30.021046 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:30.021052 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:30.021056 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:30.021151 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:30.021534 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:30.021541 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:30.021546 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:30.021816 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:30.021928 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:30.022481 (+ 553us) server_negotiation.cc:299] Negotiation successful
0805 23:10:30.022653 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260805 23:10:30.023840 29924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:30.009264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:38007 (local address 127.25.92.194:58589)
0805 23:10:30.009749 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:30.009792 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:30.011156 (+ 1364us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:30.011728 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:30.011734 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:30.012223 (+ 489us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:30.012856 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:30.012872 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:30.014015 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:30.014018 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:30.014947 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:30.014958 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:30.015531 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:30.015540 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:30.015758 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:30.016394 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:10:30.016410 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:30.018259 (+ 1849us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:10:30.020648 (+ 2389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:30.020653 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:30.020667 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:30.020933 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:30.021265 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:30.021268 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:30.021269 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:30.021402 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:30.021958 (+ 556us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:10:30.021964 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:10:30.022249 (+ 285us) client_negotiation.cc:769] Sending connection context
0805 23:10:30.022431 (+ 182us) client_negotiation.cc:240] Negotiation successful
0805 23:10:30.022852 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":103,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.92.195@KRBTEST.COM" created.
I20260805 23:10:30.025259 29922 heartbeater.cc:347] Connected to a master server at 127.25.92.254:38007
I20260805 23:10:30.025552 29922 heartbeater.cc:464] Registering TS with master...
I20260805 23:10:30.026065 29922 heartbeater.cc:511] Master 127.25.92.254:38007 requested a full tablet report, sending...
I20260805 23:10:30.027217 29592 ts_manager.cc:194] Registered new tserver with Master: b94436eb1acc4f648a487633a453e7d1 (127.25.92.194:39103)
I20260805 23:10:30.028111 29592 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.92.194@KRBTEST.COM'} at 127.25.92.194:58589
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.92.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.195.keytab.
Entry for principal kudu/127.25.92.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.195.keytab.
I20260805 23:10:30.041157 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971406251336-25971-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971406251336-25971-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971406251336-25971-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971406251336-25971-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.92.195:0
--local_ip_for_outbound_sockets=127.25.92.195
--webserver_interface=127.25.92.195
--webserver_port=0
--tserver_master_addrs=127.25.92.254:38007
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.195.keytab
--principal=kudu/127.25.92.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:34037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:10:30.157325 29927 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:10:30.157696 29927 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:10:30.157774 29927 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:10:30.162112 29927 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:10:30.162195 29927 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:10:30.162299 29927 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.92.195
I20260805 23:10:30.168164 29927 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:34037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971406251336-25971-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971406251336-25971-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.195.keytab
--principal=kudu/127.25.92.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.92.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971406251336-25971-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.92.195
--webserver_port=0
--tserver_master_addrs=127.25.92.254:38007
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29927
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.92.195
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971406251336-25971-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:10:30.169312 29927 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:10:30.170533 29927 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:10:30.180323 29933 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:30.180316 29932 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:30.180325 29935 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:10:30.181248 29927 server_base.cc:1034] running on GCE node
I20260805 23:10:30.181756 29927 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:10:30.182487 29927 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:10:30.183674 29927 hybrid_clock.cc:648] HybridClock initialized: now 1785971430183636 us; error 52 us; skew 500 ppm
Aug 05 23:10:30 dist-test-slave-ss4n krb5kdc[29547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971430, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:10:30.186698 29927 init.cc:377] Logged in from keytab as kudu/127.25.92.195@KRBTEST.COM (short username kudu)
I20260805 23:10:30.187650 29927 webserver.cc:460] Webserver started at http://127.25.92.195:38087/ using document root <none> and password file <none>
I20260805 23:10:30.188246 29927 fs_manager.cc:359] Metadata directory not provided
I20260805 23:10:30.188336 29927 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:10:30.188556 29927 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:10:30.190753 29927 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971406251336-25971-0/minicluster-data/ts-2/data/instance:
uuid: "78165dc550be406bac836802f7c75ddd"
format_stamp: "Formatted at 2026-08-05 23:10:30 on dist-test-slave-ss4n"
server_key: "bfc02d86381c9b758fa1c8466f553ab9"
server_key_iv: "5857a7d7efa1963fd1db764edcfb724f"
server_key_version: "encryptionkey@0"
I20260805 23:10:30.191352 29927 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971406251336-25971-0/minicluster-data/ts-2/wal/instance:
uuid: "78165dc550be406bac836802f7c75ddd"
format_stamp: "Formatted at 2026-08-05 23:10:30 on dist-test-slave-ss4n"
server_key: "bfc02d86381c9b758fa1c8466f553ab9"
server_key_iv: "5857a7d7efa1963fd1db764edcfb724f"
server_key_version: "encryptionkey@0"
I20260805 23:10:30.195437 29927 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260805 23:10:30.198074 29942 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:30.199383 29927 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:10:30.199523 29927 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971406251336-25971-0/minicluster-data/ts-2/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971406251336-25971-0/minicluster-data/ts-2/wal
uuid: "78165dc550be406bac836802f7c75ddd"
format_stamp: "Formatted at 2026-08-05 23:10:30 on dist-test-slave-ss4n"
server_key: "bfc02d86381c9b758fa1c8466f553ab9"
server_key_iv: "5857a7d7efa1963fd1db764edcfb724f"
server_key_version: "encryptionkey@0"
I20260805 23:10:30.199648 29927 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971406251336-25971-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971406251336-25971-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971406251336-25971-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:10:30.210785 29927 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:10:30.213910 29927 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:10:30.214115 29927 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:10:30.214725 29927 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:10:30.215685 29927 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:10:30.215765 29927 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:30.215852 29927 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:10:30.215898 29927 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:30.225566 29927 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.195:41841
I20260805 23:10:30.225682 30056 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.195:41841 every 8 connection(s)
I20260805 23:10:30.226540 29927 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971406251336-25971-0/minicluster-data/ts-2/data/info.pb
I20260805 23:10:30.227938 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 29927
I20260805 23:10:30.228072 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971406251336-25971-0/minicluster-data/ts-2/wal/instance
I20260805 23:10:30.228422 25971 external_mini_cluster.cc:1428] Setting key 95ea07ac1236b15fa58be26c457f1093
Aug 05 23:10:30 dist-test-slave-ss4n krb5kdc[29547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971430, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.195@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM
I20260805 23:10:30.241387 29631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:30.228595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.195:35629 (local address 127.25.92.254:38007)
0805 23:10:30.228745 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:30.228751 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:30.229854 (+ 1103us) server_negotiation.cc:407] Connection header received
0805 23:10:30.231586 (+ 1732us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:30.231588 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:30.231638 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:30.231714 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:30.232955 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:30.233535 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:30.234767 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:30.235011 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:30.235095 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:30.237858 (+ 2763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:30.237861 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:30.237863 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:30.237888 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:30.239254 (+ 1366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:30.239813 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:30.239815 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:30.239816 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:30.239892 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:30.240252 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:30.240254 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:30.240255 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:30.240432 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:30.240508 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:30.241079 (+ 571us) server_negotiation.cc:299] Negotiation successful
0805 23:10:30.241219 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260805 23:10:30.242430 30059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:30.229014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:38007 (local address 127.25.92.195:35629)
0805 23:10:30.229675 (+ 661us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:30.229741 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:30.231200 (+ 1459us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:30.231892 (+ 692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:30.231898 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:30.232266 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:30.232810 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:30.232824 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:30.233689 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:30.233692 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:30.234652 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:30.234662 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:30.235177 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:30.235180 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:30.235334 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:30.236077 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:10:30.236093 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:30.237674 (+ 1581us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:10:30.239374 (+ 1700us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:30.239379 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:30.239392 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:30.239721 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:30.240000 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:30.240003 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:30.240005 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:30.240162 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:30.240523 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:10:30.240528 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:10:30.240803 (+ 275us) client_negotiation.cc:769] Sending connection context
0805 23:10:30.241026 (+ 223us) client_negotiation.cc:240] Negotiation successful
0805 23:10:30.241378 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":145,"threads_started":1}
I20260805 23:10:30.243497 30057 heartbeater.cc:347] Connected to a master server at 127.25.92.254:38007
I20260805 23:10:30.243772 30057 heartbeater.cc:464] Registering TS with master...
I20260805 23:10:30.244283 30057 heartbeater.cc:511] Master 127.25.92.254:38007 requested a full tablet report, sending...
I20260805 23:10:30.245328 29592 ts_manager.cc:194] Registered new tserver with Master: 78165dc550be406bac836802f7c75ddd (127.25.92.195:41841)
I20260805 23:10:30.245950 29592 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.92.195@KRBTEST.COM'} at 127.25.92.195:35629
I20260805 23:10:30.255247 25971 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 05 23:10:30 dist-test-slave-ss4n krb5kdc[29547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971430, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 05 23:10:30 dist-test-slave-ss4n krb5kdc[29547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971430, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.92.193@KRBTEST.COM
I20260805 23:10:30.284363 30064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:30.273453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49732 (local address 127.25.92.193:36487)
0805 23:10:30.273834 (+ 381us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:30.273857 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:30.273896 (+ 39us) server_negotiation.cc:407] Connection header received
0805 23:10:30.273956 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:30.273961 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:30.274179 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:30.274325 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:30.275053 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:30.275989 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:30.276842 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:30.277305 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:30.277404 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:30.280308 (+ 2904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:30.280320 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:30.280336 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:30.280365 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:30.282399 (+ 2034us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:30.282819 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:30.282824 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:30.282826 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:30.282922 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:30.283252 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:30.283255 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:30.283256 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:30.283575 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:30.283753 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:30.284042 (+ 289us) server_negotiation.cc:299] Negotiation successful
0805 23:10:30.284186 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":270,"thread_start_us":155,"threads_started":1}
Aug 05 23:10:30 dist-test-slave-ss4n krb5kdc[29547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971430, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM
I20260805 23:10:30.296747 29631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:30.287868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51430 (local address 127.25.92.254:38007)
0805 23:10:30.288048 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:30.288052 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:30.288156 (+ 104us) server_negotiation.cc:407] Connection header received
0805 23:10:30.288228 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:30.288230 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:30.288274 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:30.288338 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:30.289109 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:30.289763 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:30.290518 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:30.290742 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:30.290834 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:30.293568 (+ 2734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:30.293572 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:30.293575 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:30.293604 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:30.295111 (+ 1507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:30.295541 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:30.295543 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:30.295545 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:30.295620 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:30.295870 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:30.295873 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:30.295874 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:30.296102 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:30.296226 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:30.296441 (+ 215us) server_negotiation.cc:299] Negotiation successful
0805 23:10:30.296545 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
W20260805 23:10:30.297238 29592 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51430
I20260805 23:10:30.297981 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 29654
I20260805 23:10:30.304913 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 29792
I20260805 23:10:30.310678 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 29927
I20260805 23:10:30.316447 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 29560
2026-08-05T23:10:30Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2333 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971406251336-25971-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 23:10:30 dist-test-slave-ss4n krb5kdc[30069](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 23:10:30 dist-test-slave-ss4n krb5kdc[30069](info): set up 2 sockets
Aug 05 23:10:30 dist-test-slave-ss4n krb5kdc[30069](info): commencing operation
krb5kdc: starting...
W20260805 23:10:31.400988 25971 mini_kdc.cc:121] Time spent starting KDC: real 1.062s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971406251336-25971-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971406251336-25971-0/krb5kdc/test-user.keytab.
Aug 05 23:10:31 dist-test-slave-ss4n krb5kdc[30069](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971431, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T23:10:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:10:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.92.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.92.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.92.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab.
Entry for principal kudu/127.25.92.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab.
I20260805 23:10:31.530081 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971406251336-25971-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971406251336-25971-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971406251336-25971-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.254:46003
--webserver_interface=127.25.92.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab
--principal=kudu/127.25.92.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:35211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.92.254:46003
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:10:31.644202 30083 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:10:31.644553 30083 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:10:31.644642 30083 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:10:31.648844 30083 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:10:31.648949 30083 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:10:31.648993 30083 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:10:31.649029 30083 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:10:31.649070 30083 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:10:31.654697 30083 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:35211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971406251336-25971-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971406251336-25971-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.92.254:46003
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab
--principal=kudu/127.25.92.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.254:46003
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.92.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30083
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971406251336-25971-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:10:31.655859 30083 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:10:31.656904 30083 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:10:31.666018 30091 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:31.666006 30088 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:10:31.666270 30083 server_base.cc:1034] running on GCE node
W20260805 23:10:31.666028 30089 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:10:31.667111 30083 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:10:31.668339 30083 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:10:31.669556 30083 hybrid_clock.cc:648] HybridClock initialized: now 1785971431669528 us; error 50 us; skew 500 ppm
Aug 05 23:10:31 dist-test-slave-ss4n krb5kdc[30069](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971431, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:10:31.672526 30083 init.cc:377] Logged in from keytab as kudu/127.25.92.254@KRBTEST.COM (short username kudu)
I20260805 23:10:31.673332 30083 fs_manager.cc:359] Metadata directory not provided
I20260805 23:10:31.673434 30083 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:10:31.673729 30083 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:10:31.675884 30083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971406251336-25971-0/minicluster-data/master-0/data/instance:
uuid: "e358b140f5304d1d9b5356a045bd1635"
format_stamp: "Formatted at 2026-08-05 23:10:31 on dist-test-slave-ss4n"
server_key: "6d7434048419320eab7b0a5c50dcd975"
server_key_iv: "8ff0c8207ef3aea6491760fb320ce584"
server_key_version: "encryptionkey@0"
I20260805 23:10:31.676537 30083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971406251336-25971-0/minicluster-data/master-0/wal/instance:
uuid: "e358b140f5304d1d9b5356a045bd1635"
format_stamp: "Formatted at 2026-08-05 23:10:31 on dist-test-slave-ss4n"
server_key: "6d7434048419320eab7b0a5c50dcd975"
server_key_iv: "8ff0c8207ef3aea6491760fb320ce584"
server_key_version: "encryptionkey@0"
I20260805 23:10:31.680496 30083 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260805 23:10:31.683442 30097 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:31.684804 30083 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260805 23:10:31.684981 30083 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971406251336-25971-0/minicluster-data/master-0/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971406251336-25971-0/minicluster-data/master-0/wal
uuid: "e358b140f5304d1d9b5356a045bd1635"
format_stamp: "Formatted at 2026-08-05 23:10:31 on dist-test-slave-ss4n"
server_key: "6d7434048419320eab7b0a5c50dcd975"
server_key_iv: "8ff0c8207ef3aea6491760fb320ce584"
server_key_version: "encryptionkey@0"
I20260805 23:10:31.685113 30083 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971406251336-25971-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971406251336-25971-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971406251336-25971-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:10:31.707911 30083 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:10:31.711827 30083 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:10:31.712073 30083 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:10:31.719981 30083 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.254:46003
I20260805 23:10:31.720285 30150 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.254:46003 every 8 connection(s)
I20260805 23:10:31.720733 30083 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
I20260805 23:10:31.723615 30151 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:10:31.727092 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 30083
I20260805 23:10:31.727229 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971406251336-25971-0/minicluster-data/master-0/wal/instance
I20260805 23:10:31.727540 25971 external_mini_cluster.cc:1428] Setting key 475e1e2eae331824815120767af6f35f
I20260805 23:10:31.729910 30151 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e358b140f5304d1d9b5356a045bd1635: Bootstrap starting.
I20260805 23:10:31.732115 30151 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e358b140f5304d1d9b5356a045bd1635: Neither blocks nor log segments found. Creating new log.
I20260805 23:10:31.732856 30151 log.cc:824] T 00000000000000000000000000000000 P e358b140f5304d1d9b5356a045bd1635: Log is configured to *not* fsync() on all Append() calls
I20260805 23:10:31.734783 30151 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e358b140f5304d1d9b5356a045bd1635: No bootstrap required, opened a new log
Aug 05 23:10:31 dist-test-slave-ss4n krb5kdc[30069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971431, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM
I20260805 23:10:31.738566 30151 raft_consensus.cc:348] T 00000000000000000000000000000000 P e358b140f5304d1d9b5356a045bd1635 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e358b140f5304d1d9b5356a045bd1635" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 46003 } }
I20260805 23:10:31.738821 30151 raft_consensus.cc:374] T 00000000000000000000000000000000 P e358b140f5304d1d9b5356a045bd1635 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:10:31.738904 30151 raft_consensus.cc:729] T 00000000000000000000000000000000 P e358b140f5304d1d9b5356a045bd1635 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e358b140f5304d1d9b5356a045bd1635, State: Initialized, Role: FOLLOWER
I20260805 23:10:31.739418 30151 consensus_queue.cc:260] T 00000000000000000000000000000000 P e358b140f5304d1d9b5356a045bd1635 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e358b140f5304d1d9b5356a045bd1635" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 46003 } }
I20260805 23:10:31.739570 30151 raft_consensus.cc:388] T 00000000000000000000000000000000 P e358b140f5304d1d9b5356a045bd1635 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:10:31.739656 30151 raft_consensus.cc:482] T 00000000000000000000000000000000 P e358b140f5304d1d9b5356a045bd1635 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:10:31.739771 30151 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e358b140f5304d1d9b5356a045bd1635 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:10:31.741092 30151 raft_consensus.cc:504] T 00000000000000000000000000000000 P e358b140f5304d1d9b5356a045bd1635 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e358b140f5304d1d9b5356a045bd1635" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 46003 } }
I20260805 23:10:31.741472 30151 leader_election.cc:302] T 00000000000000000000000000000000 P e358b140f5304d1d9b5356a045bd1635 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e358b140f5304d1d9b5356a045bd1635; no voters:
I20260805 23:10:31.741858 30151 leader_election.cc:288] T 00000000000000000000000000000000 P e358b140f5304d1d9b5356a045bd1635 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 23:10:31.742005 30156 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e358b140f5304d1d9b5356a045bd1635 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:10:31.742282 30156 raft_consensus.cc:686] T 00000000000000000000000000000000 P e358b140f5304d1d9b5356a045bd1635 [term 1 LEADER]: Becoming Leader. State: Replica: e358b140f5304d1d9b5356a045bd1635, State: Running, Role: LEADER
I20260805 23:10:31.742726 30156 consensus_queue.cc:237] T 00000000000000000000000000000000 P e358b140f5304d1d9b5356a045bd1635 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e358b140f5304d1d9b5356a045bd1635" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 46003 } }
I20260805 23:10:31.742784 30154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:31.729097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51854 (local address 127.25.92.254:46003)
0805 23:10:31.729637 (+ 540us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:31.729655 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:31.729693 (+ 38us) server_negotiation.cc:407] Connection header received
0805 23:10:31.730428 (+ 735us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:31.730448 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:31.730779 (+ 331us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:31.731126 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:31.731943 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:31.733047 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:31.733939 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:31.734407 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:31.734581 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:31.737288 (+ 2707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:31.737292 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:31.737305 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:31.737331 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:31.739259 (+ 1928us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:31.739731 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:31.739737 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:31.739744 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:31.739935 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:31.740242 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:31.740245 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:31.740247 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:31.740797 (+ 550us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:31.741018 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:31.741421 (+ 403us) server_negotiation.cc:299] Negotiation successful
0805 23:10:31.741786 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":136,"threads_started":1}
I20260805 23:10:31.743206 30151 sys_catalog.cc:564] T 00000000000000000000000000000000 P e358b140f5304d1d9b5356a045bd1635 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:10:31.745886 30157 sys_catalog.cc:455] T 00000000000000000000000000000000 P e358b140f5304d1d9b5356a045bd1635 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e358b140f5304d1d9b5356a045bd1635" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e358b140f5304d1d9b5356a045bd1635" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 46003 } } }
I20260805 23:10:31.746049 30157 sys_catalog.cc:458] T 00000000000000000000000000000000 P e358b140f5304d1d9b5356a045bd1635 [sys.catalog]: This master's current role is: LEADER
I20260805 23:10:31.746380 30158 sys_catalog.cc:455] T 00000000000000000000000000000000 P e358b140f5304d1d9b5356a045bd1635 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e358b140f5304d1d9b5356a045bd1635. Latest consensus state: current_term: 1 leader_uuid: "e358b140f5304d1d9b5356a045bd1635" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e358b140f5304d1d9b5356a045bd1635" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 46003 } } }
I20260805 23:10:31.746505 30158 sys_catalog.cc:458] T 00000000000000000000000000000000 P e358b140f5304d1d9b5356a045bd1635 [sys.catalog]: This master's current role is: LEADER
W20260805 23:10:31.748768 30170 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e358b140f5304d1d9b5356a045bd1635: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260805 23:10:31.748852 30170 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260805 23:10:31.749001 30172 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:10:31.749989 30172 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:10:31.756292 30172 catalog_manager.cc:1347] Generated new cluster ID: 793382b3d7ca44e3b372e593a15added
I20260805 23:10:31.756369 30172 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:10:31.772403 30172 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:10:31.773727 30172 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:10:31.780319 30172 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e358b140f5304d1d9b5356a045bd1635: Generated new TSK 0
I20260805 23:10:31.781075 30172 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.92.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.92.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.92.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.193.keytab.
Entry for principal kudu/127.25.92.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.193.keytab.
I20260805 23:10:31.816164 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971406251336-25971-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971406251336-25971-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971406251336-25971-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971406251336-25971-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.92.193:0
--local_ip_for_outbound_sockets=127.25.92.193
--webserver_interface=127.25.92.193
--webserver_port=0
--tserver_master_addrs=127.25.92.254:46003
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.193.keytab
--principal=kudu/127.25.92.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:35211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:10:31.937460 30177 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:10:31.937813 30177 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:10:31.937892 30177 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:10:31.942121 30177 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:10:31.942209 30177 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:10:31.942312 30177 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.92.193
I20260805 23:10:31.948083 30177 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:35211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971406251336-25971-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971406251336-25971-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.193.keytab
--principal=kudu/127.25.92.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.92.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971406251336-25971-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.92.193
--webserver_port=0
--tserver_master_addrs=127.25.92.254:46003
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30177
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.92.193
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971406251336-25971-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:10:31.949229 30177 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:10:31.950331 30177 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:10:31.960049 30182 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:31.960073 30183 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:31.960042 30185 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:10:31.960302 30177 server_base.cc:1034] running on GCE node
I20260805 23:10:31.960824 30177 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:10:31.961632 30177 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:10:31.962813 30177 hybrid_clock.cc:648] HybridClock initialized: now 1785971431962794 us; error 34 us; skew 500 ppm
Aug 05 23:10:31 dist-test-slave-ss4n krb5kdc[30069](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971431, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:10:31.965859 30177 init.cc:377] Logged in from keytab as kudu/127.25.92.193@KRBTEST.COM (short username kudu)
I20260805 23:10:31.966620 30177 fs_manager.cc:359] Metadata directory not provided
I20260805 23:10:31.966688 30177 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:10:31.966861 30177 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:10:31.969002 30177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971406251336-25971-0/minicluster-data/ts-0/data/instance:
uuid: "694df89106e64d9abd5b7e1fe899776a"
format_stamp: "Formatted at 2026-08-05 23:10:31 on dist-test-slave-ss4n"
server_key: "2cef230b10c4c9e9758163a4397ab64f"
server_key_iv: "cf51d434979ea3b3143c3436b909418b"
server_key_version: "encryptionkey@0"
I20260805 23:10:31.969599 30177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971406251336-25971-0/minicluster-data/ts-0/wal/instance:
uuid: "694df89106e64d9abd5b7e1fe899776a"
format_stamp: "Formatted at 2026-08-05 23:10:31 on dist-test-slave-ss4n"
server_key: "2cef230b10c4c9e9758163a4397ab64f"
server_key_iv: "cf51d434979ea3b3143c3436b909418b"
server_key_version: "encryptionkey@0"
I20260805 23:10:31.973414 30177 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.000s
I20260805 23:10:31.975895 30191 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:31.977238 30177 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260805 23:10:31.977363 30177 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971406251336-25971-0/minicluster-data/ts-0/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971406251336-25971-0/minicluster-data/ts-0/wal
uuid: "694df89106e64d9abd5b7e1fe899776a"
format_stamp: "Formatted at 2026-08-05 23:10:31 on dist-test-slave-ss4n"
server_key: "2cef230b10c4c9e9758163a4397ab64f"
server_key_iv: "cf51d434979ea3b3143c3436b909418b"
server_key_version: "encryptionkey@0"
I20260805 23:10:31.977471 30177 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971406251336-25971-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971406251336-25971-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971406251336-25971-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:10:32.010102 30177 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:10:32.013531 30177 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:10:32.013814 30177 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:10:32.014467 30177 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:10:32.015334 30177 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:10:32.015383 30177 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:32.015450 30177 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:10:32.015482 30177 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:32.024587 30177 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.193:46277
I20260805 23:10:32.024652 30306 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.193:46277 every 8 connection(s)
I20260805 23:10:32.025241 30177 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971406251336-25971-0/minicluster-data/ts-0/data/info.pb
I20260805 23:10:32.034337 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 30177
I20260805 23:10:32.034464 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971406251336-25971-0/minicluster-data/ts-0/wal/instance
I20260805 23:10:32.034829 25971 external_mini_cluster.cc:1428] Setting key 06c509213aeee3c35fab498e13509c65
Aug 05 23:10:32 dist-test-slave-ss4n krb5kdc[30069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971431, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.193@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM
I20260805 23:10:32.040375 30154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:32.027148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.193:34623 (local address 127.25.92.254:46003)
0805 23:10:32.027330 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:32.027335 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:32.028539 (+ 1204us) server_negotiation.cc:407] Connection header received
0805 23:10:32.029375 (+ 836us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:32.029378 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:32.029427 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:32.029561 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:32.030738 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:32.031280 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:32.032611 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:32.032838 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:32.032917 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:32.035965 (+ 3048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:32.035968 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:32.035970 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:32.035994 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:32.037954 (+ 1960us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:32.038595 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:32.038599 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:32.038601 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:32.038710 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:32.039097 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:32.039100 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:32.039102 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:32.039321 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:32.039432 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:32.040037 (+ 605us) server_negotiation.cc:299] Negotiation successful
0805 23:10:32.040201 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260805 23:10:32.041301 30309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:32.027562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:46003 (local address 127.25.92.193:34623)
0805 23:10:32.028388 (+ 826us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:32.028431 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:32.029195 (+ 764us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:32.029723 (+ 528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:32.029729 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:32.030098 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:32.030585 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:32.030599 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:32.031423 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:32.031426 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:32.032485 (+ 1059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:32.032495 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:32.033000 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:32.033004 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:32.033166 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:32.033888 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:10:32.033906 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:32.035793 (+ 1887us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:10:32.038099 (+ 2306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:32.038104 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:32.038119 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:32.038403 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:32.038821 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:32.038824 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:32.038825 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:32.038977 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:32.039433 (+ 456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:10:32.039441 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:10:32.039785 (+ 344us) client_negotiation.cc:769] Sending connection context
0805 23:10:32.040015 (+ 230us) client_negotiation.cc:240] Negotiation successful
0805 23:10:32.040360 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":121,"threads_started":1}
I20260805 23:10:32.042802 30307 heartbeater.cc:347] Connected to a master server at 127.25.92.254:46003
I20260805 23:10:32.043073 30307 heartbeater.cc:464] Registering TS with master...
I20260805 23:10:32.043542 30307 heartbeater.cc:511] Master 127.25.92.254:46003 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.25.92.194@KRBTEST.COM; defaulting to no policy
I20260805 23:10:32.045400 30114 ts_manager.cc:194] Registered new tserver with Master: 694df89106e64d9abd5b7e1fe899776a (127.25.92.193:46277)
I20260805 23:10:32.046787 30114 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.92.193@KRBTEST.COM'} at 127.25.92.193:34623
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.92.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.92.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.194.keytab.
Entry for principal kudu/127.25.92.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.194.keytab.
I20260805 23:10:32.065333 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971406251336-25971-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971406251336-25971-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971406251336-25971-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971406251336-25971-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.92.194:0
--local_ip_for_outbound_sockets=127.25.92.194
--webserver_interface=127.25.92.194
--webserver_port=0
--tserver_master_addrs=127.25.92.254:46003
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.194.keytab
--principal=kudu/127.25.92.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:35211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:10:32.180835 30312 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:10:32.181142 30312 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:10:32.181227 30312 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:10:32.185585 30312 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:10:32.185670 30312 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:10:32.185763 30312 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.92.194
I20260805 23:10:32.191500 30312 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:35211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971406251336-25971-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971406251336-25971-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.194.keytab
--principal=kudu/127.25.92.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.92.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971406251336-25971-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.92.194
--webserver_port=0
--tserver_master_addrs=127.25.92.254:46003
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30312
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.92.194
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971406251336-25971-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:10:32.192632 30312 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:10:32.193696 30312 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:10:32.203497 30320 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:10:32.203601 30312 server_base.cc:1034] running on GCE node
W20260805 23:10:32.203512 30318 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:32.203497 30317 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:10:32.204288 30312 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:10:32.204887 30312 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:10:32.206077 30312 hybrid_clock.cc:648] HybridClock initialized: now 1785971432206035 us; error 49 us; skew 500 ppm
Aug 05 23:10:32 dist-test-slave-ss4n krb5kdc[30069](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971432, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:10:32.208972 30312 init.cc:377] Logged in from keytab as kudu/127.25.92.194@KRBTEST.COM (short username kudu)
I20260805 23:10:32.209779 30312 fs_manager.cc:359] Metadata directory not provided
I20260805 23:10:32.209872 30312 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:10:32.210111 30312 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:10:32.212164 30312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971406251336-25971-0/minicluster-data/ts-1/data/instance:
uuid: "5e393127ce344afd90b6d5a0677dacd2"
format_stamp: "Formatted at 2026-08-05 23:10:32 on dist-test-slave-ss4n"
server_key: "a5722ae3fd8b58bd3090896f27746c67"
server_key_iv: "acca8bf2a4d22e02260639203833142d"
server_key_version: "encryptionkey@0"
I20260805 23:10:32.212738 30312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971406251336-25971-0/minicluster-data/ts-1/wal/instance:
uuid: "5e393127ce344afd90b6d5a0677dacd2"
format_stamp: "Formatted at 2026-08-05 23:10:32 on dist-test-slave-ss4n"
server_key: "a5722ae3fd8b58bd3090896f27746c67"
server_key_iv: "acca8bf2a4d22e02260639203833142d"
server_key_version: "encryptionkey@0"
I20260805 23:10:32.216564 30312 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260805 23:10:32.219128 30326 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:32.220530 30312 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260805 23:10:32.220752 30312 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971406251336-25971-0/minicluster-data/ts-1/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971406251336-25971-0/minicluster-data/ts-1/wal
uuid: "5e393127ce344afd90b6d5a0677dacd2"
format_stamp: "Formatted at 2026-08-05 23:10:32 on dist-test-slave-ss4n"
server_key: "a5722ae3fd8b58bd3090896f27746c67"
server_key_iv: "acca8bf2a4d22e02260639203833142d"
server_key_version: "encryptionkey@0"
I20260805 23:10:32.220888 30312 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971406251336-25971-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971406251336-25971-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971406251336-25971-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:10:32.257822 30312 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:10:32.261451 30312 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:10:32.261740 30312 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:10:32.262471 30312 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:10:32.263363 30312 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:10:32.263438 30312 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:32.263513 30312 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:10:32.263574 30312 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:32.272957 30312 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.194:35309
I20260805 23:10:32.272975 30441 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.194:35309 every 8 connection(s)
I20260805 23:10:32.273649 30312 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971406251336-25971-0/minicluster-data/ts-1/data/info.pb
I20260805 23:10:32.283069 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 30312
I20260805 23:10:32.283215 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971406251336-25971-0/minicluster-data/ts-1/wal/instance
I20260805 23:10:32.283543 25971 external_mini_cluster.cc:1428] Setting key 8f5800c9d7a172971abaa3450d5e464d
Aug 05 23:10:32 dist-test-slave-ss4n krb5kdc[30069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971432, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.194@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM
I20260805 23:10:32.289534 30154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:32.275572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.194:59989 (local address 127.25.92.254:46003)
0805 23:10:32.275792 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:32.275797 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:32.276777 (+ 980us) server_negotiation.cc:407] Connection header received
0805 23:10:32.278066 (+ 1289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:32.278069 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:32.278120 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:32.278206 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:32.279387 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:32.280207 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:32.281454 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:32.281746 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:32.281904 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:32.285004 (+ 3100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:32.285007 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:32.285009 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:32.285032 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:32.286998 (+ 1966us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:32.287760 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:32.287762 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:32.287763 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:32.287845 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:32.288253 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:32.288255 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:32.288256 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:32.288482 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:32.288561 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:32.289211 (+ 650us) server_negotiation.cc:299] Negotiation successful
0805 23:10:32.289324 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115}
I20260805 23:10:32.290800 30444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:32.276133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:46003 (local address 127.25.92.194:59989)
0805 23:10:32.276610 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:32.276656 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:32.277865 (+ 1209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:32.278369 (+ 504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:32.278375 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:32.278741 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:32.279231 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:32.279245 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:32.280355 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:32.280360 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:32.281337 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:32.281347 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:32.282049 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:32.282053 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:32.282222 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:32.282913 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:10:32.282934 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:32.284820 (+ 1886us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:10:32.287147 (+ 2327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:32.287156 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:32.287179 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:32.287656 (+ 477us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:32.287935 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:32.287939 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:32.287942 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:32.288141 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:32.288585 (+ 444us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:10:32.288594 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:10:32.288942 (+ 348us) client_negotiation.cc:769] Sending connection context
0805 23:10:32.289162 (+ 220us) client_negotiation.cc:240] Negotiation successful
0805 23:10:32.289602 (+ 440us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":123,"threads_started":1}
I20260805 23:10:32.292109 30442 heartbeater.cc:347] Connected to a master server at 127.25.92.254:46003
I20260805 23:10:32.292379 30442 heartbeater.cc:464] Registering TS with master...
I20260805 23:10:32.292851 30442 heartbeater.cc:511] Master 127.25.92.254:46003 requested a full tablet report, sending...
I20260805 23:10:32.294009 30114 ts_manager.cc:194] Registered new tserver with Master: 5e393127ce344afd90b6d5a0677dacd2 (127.25.92.194:35309)
WARNING: no policy specified for kudu/127.25.92.195@KRBTEST.COM; defaulting to no policy
I20260805 23:10:32.294642 30114 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.92.194@KRBTEST.COM'} at 127.25.92.194:59989
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.92.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.92.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.195.keytab.
Entry for principal kudu/127.25.92.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.195.keytab.
I20260805 23:10:32.312876 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971406251336-25971-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971406251336-25971-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971406251336-25971-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971406251336-25971-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.92.195:0
--local_ip_for_outbound_sockets=127.25.92.195
--webserver_interface=127.25.92.195
--webserver_port=0
--tserver_master_addrs=127.25.92.254:46003
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.195.keytab
--principal=kudu/127.25.92.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:35211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:10:32.430052 30447 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:10:32.430362 30447 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:10:32.430464 30447 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:10:32.434610 30447 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:10:32.434698 30447 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:10:32.434823 30447 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.92.195
I20260805 23:10:32.440440 30447 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:35211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971406251336-25971-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971406251336-25971-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.195.keytab
--principal=kudu/127.25.92.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.92.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971406251336-25971-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.92.195
--webserver_port=0
--tserver_master_addrs=127.25.92.254:46003
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30447
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.92.195
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971406251336-25971-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:10:32.441644 30447 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:10:32.442700 30447 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:10:32.453059 30455 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:32.453061 30452 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:10:32.453114 30447 server_base.cc:1034] running on GCE node
W20260805 23:10:32.453061 30453 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:10:32.453908 30447 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:10:32.454592 30447 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:10:32.455756 30447 hybrid_clock.cc:648] HybridClock initialized: now 1785971432455738 us; error 31 us; skew 500 ppm
Aug 05 23:10:32 dist-test-slave-ss4n krb5kdc[30069](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971432, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:10:32.458683 30447 init.cc:377] Logged in from keytab as kudu/127.25.92.195@KRBTEST.COM (short username kudu)
I20260805 23:10:32.459468 30447 fs_manager.cc:359] Metadata directory not provided
I20260805 23:10:32.459532 30447 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:10:32.459717 30447 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:10:32.461833 30447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971406251336-25971-0/minicluster-data/ts-2/data/instance:
uuid: "8c9dc7929d514cfb86d62699cd53dd1b"
format_stamp: "Formatted at 2026-08-05 23:10:32 on dist-test-slave-ss4n"
server_key: "10363dc858d2e56f6e2ba1904612d93c"
server_key_iv: "b73c1978fd6f4cd04507917bfd0020dc"
server_key_version: "encryptionkey@0"
I20260805 23:10:32.462373 30447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971406251336-25971-0/minicluster-data/ts-2/wal/instance:
uuid: "8c9dc7929d514cfb86d62699cd53dd1b"
format_stamp: "Formatted at 2026-08-05 23:10:32 on dist-test-slave-ss4n"
server_key: "10363dc858d2e56f6e2ba1904612d93c"
server_key_iv: "b73c1978fd6f4cd04507917bfd0020dc"
server_key_version: "encryptionkey@0"
I20260805 23:10:32.466079 30447 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260805 23:10:32.468524 30461 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:32.469928 30447 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:10:32.470049 30447 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971406251336-25971-0/minicluster-data/ts-2/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971406251336-25971-0/minicluster-data/ts-2/wal
uuid: "8c9dc7929d514cfb86d62699cd53dd1b"
format_stamp: "Formatted at 2026-08-05 23:10:32 on dist-test-slave-ss4n"
server_key: "10363dc858d2e56f6e2ba1904612d93c"
server_key_iv: "b73c1978fd6f4cd04507917bfd0020dc"
server_key_version: "encryptionkey@0"
I20260805 23:10:32.470162 30447 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971406251336-25971-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971406251336-25971-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971406251336-25971-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:10:32.483204 30447 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:10:32.486505 30447 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:10:32.486711 30447 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:10:32.487296 30447 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:10:32.488140 30447 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:10:32.488191 30447 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:32.488255 30447 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:10:32.488292 30447 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:32.497375 30447 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.195:45393
I20260805 23:10:32.497396 30577 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.195:45393 every 8 connection(s)
I20260805 23:10:32.497979 30447 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971406251336-25971-0/minicluster-data/ts-2/data/info.pb
I20260805 23:10:32.499635 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 30447
I20260805 23:10:32.499810 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971406251336-25971-0/minicluster-data/ts-2/wal/instance
I20260805 23:10:32.500102 25971 external_mini_cluster.cc:1428] Setting key 3a1c17e272f8cf4544018bba6c38f316
Aug 05 23:10:32 dist-test-slave-ss4n krb5kdc[30069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971432, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.195@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM
I20260805 23:10:32.511402 30154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:32.499736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.195:54769 (local address 127.25.92.254:46003)
0805 23:10:32.500516 (+ 780us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:32.500523 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:32.500797 (+ 274us) server_negotiation.cc:407] Connection header received
0805 23:10:32.502194 (+ 1397us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:32.502198 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:32.502255 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:32.502349 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:32.503508 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:32.504051 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:32.505217 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:32.505436 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:32.505596 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:32.508134 (+ 2538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:32.508137 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:32.508139 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:32.508166 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:32.509640 (+ 1474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:32.510108 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:32.510111 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:32.510113 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:32.510189 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:32.510476 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:32.510478 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:32.510479 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:32.510648 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:32.510735 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:32.511178 (+ 443us) server_negotiation.cc:299] Negotiation successful
0805 23:10:32.511279 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":668}
I20260805 23:10:32.512342 30580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:32.500146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:46003 (local address 127.25.92.195:54769)
0805 23:10:32.500645 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:32.500699 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:32.501978 (+ 1279us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:32.502517 (+ 539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:32.502524 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:32.502871 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:32.503354 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:32.503370 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:32.504191 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:32.504199 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:32.505062 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:32.505071 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:32.505643 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:32.505647 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:32.505821 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:32.506443 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:10:32.506460 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:32.507972 (+ 1512us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:10:32.509755 (+ 1783us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:32.509760 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:32.509772 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:32.510023 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:32.510266 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:32.510268 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:32.510270 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:32.510400 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:32.510743 (+ 343us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:10:32.510748 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:10:32.510979 (+ 231us) client_negotiation.cc:769] Sending connection context
0805 23:10:32.511133 (+ 154us) client_negotiation.cc:240] Negotiation successful
0805 23:10:32.511399 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":134,"threads_started":1}
I20260805 23:10:32.513784 30578 heartbeater.cc:347] Connected to a master server at 127.25.92.254:46003
I20260805 23:10:32.514050 30578 heartbeater.cc:464] Registering TS with master...
I20260805 23:10:32.514567 30578 heartbeater.cc:511] Master 127.25.92.254:46003 requested a full tablet report, sending...
I20260805 23:10:32.515621 30113 ts_manager.cc:194] Registered new tserver with Master: 8c9dc7929d514cfb86d62699cd53dd1b (127.25.92.195:45393)
I20260805 23:10:32.516237 30113 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.92.195@KRBTEST.COM'} at 127.25.92.195:54769
I20260805 23:10:32.526513 25971 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260805 23:10:32.537263 30154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:32.528728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37236 (local address 127.25.92.254:46003)
0805 23:10:32.528902 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:32.528907 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:32.529058 (+ 151us) server_negotiation.cc:407] Connection header received
0805 23:10:32.529230 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:32.529233 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:32.529287 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:32.529383 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:32.530283 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:32.530865 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:32.531781 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:32.532031 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:32.532150 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:32.533728 (+ 1578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:32.533732 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:32.533735 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:32.533770 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:32.535401 (+ 1631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:32.535873 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:32.535878 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:32.535880 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:32.535966 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:32.536276 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:32.536280 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:32.536282 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:32.536526 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:32.536680 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:32.536938 (+ 258us) server_negotiation.cc:299] Negotiation successful
0805 23:10:32.537081 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260805 23:10:32.540588 30113 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37236:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260805 23:10:32.544211 30113 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260805 23:10:32.560310 30591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:32.553694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.194:35309 (local address 127.0.0.1:42564)
0805 23:10:32.554813 (+ 1119us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:32.554832 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:32.554928 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:32.555699 (+ 771us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:32.555702 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:32.555719 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:10:32.555930 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:32.555937 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:32.557181 (+ 1244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:32.557188 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:32.558566 (+ 1378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:32.558577 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:32.559640 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:32.559645 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:32.559764 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:32.559820 (+ 56us) client_negotiation.cc:769] Sending connection context
0805 23:10:32.559943 (+ 123us) client_negotiation.cc:240] Negotiation successful
0805 23:10:32.560098 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1052,"mutex_wait_us":126,"thread_start_us":176,"threads_started":1}
I20260805 23:10:32.560493 30590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:32.553611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60490 (local address 127.25.92.195:45393)
0805 23:10:32.555006 (+ 1395us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:32.555019 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:32.555042 (+ 23us) server_negotiation.cc:407] Connection header received
0805 23:10:32.555124 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:32.555127 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:32.555321 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:32.555482 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:10:32.555860 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:32.557102 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:32.559002 (+ 1900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:32.559766 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:32.559921 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:32.560095 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:32.560212 (+ 117us) server_negotiation.cc:299] Negotiation successful
0805 23:10:32.560314 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1289,"thread_start_us":128,"threads_started":1}
I20260805 23:10:32.561000 30594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:32.554396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42564 (local address 127.25.92.194:35309)
0805 23:10:32.554772 (+ 376us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:32.554785 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:32.555001 (+ 216us) server_negotiation.cc:407] Connection header received
0805 23:10:32.555096 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:32.555102 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:32.555332 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:32.555519 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:10:32.556085 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:32.556977 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:32.558709 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:32.559481 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:32.559616 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:32.559775 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:32.560697 (+ 922us) server_negotiation.cc:299] Negotiation successful
0805 23:10:32.560835 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273,"thread_start_us":129,"threads_started":1}
I20260805 23:10:32.560894 30589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:32.553426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.195:45393 (local address 127.0.0.1:60490)
0805 23:10:32.554128 (+ 702us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:32.554186 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:32.554327 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:32.555507 (+ 1180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:32.555513 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:32.555539 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:10:32.555723 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:32.555731 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:32.557265 (+ 1534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:32.557271 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:32.558855 (+ 1584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:32.558866 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:32.559910 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:32.559917 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:32.560059 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:32.560087 (+ 28us) client_negotiation.cc:769] Sending connection context
0805 23:10:32.560166 (+ 79us) client_negotiation.cc:240] Negotiation successful
0805 23:10:32.560252 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":534,"mutex_wait_us":194,"thread_start_us":133,"threads_started":1}
I20260805 23:10:32.561271 30593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:32.554369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50602 (local address 127.25.92.193:46277)
0805 23:10:32.554796 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:32.554805 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:32.555260 (+ 455us) server_negotiation.cc:407] Connection header received
0805 23:10:32.555361 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:32.555368 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:32.555538 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:32.555682 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:10:32.556057 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:32.557221 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:32.559107 (+ 1886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:32.559965 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:32.560089 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:32.560265 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:32.560972 (+ 707us) server_negotiation.cc:299] Negotiation successful
0805 23:10:32.561109 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":77,"threads_started":1}
I20260805 23:10:32.561599 30592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:32.553760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.193:46277 (local address 127.0.0.1:50602)
0805 23:10:32.555142 (+ 1382us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:32.555156 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:32.555256 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:32.555682 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:32.555685 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:32.555703 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:10:32.555913 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:32.555921 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:32.557334 (+ 1413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:32.557337 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:32.558976 (+ 1639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:32.558984 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:32.560336 (+ 1352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:32.560340 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:32.560436 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:32.560460 (+ 24us) client_negotiation.cc:769] Sending connection context
0805 23:10:32.560788 (+ 328us) client_negotiation.cc:240] Negotiation successful
0805 23:10:32.560875 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1319,"mutex_wait_us":534,"thread_start_us":106,"threads_started":1}
I20260805 23:10:32.564209 30374 tablet_service.cc:1467] Processing CreateTablet for tablet c05f40c4d756417b9d3acbed2b6dea62 (DEFAULT_TABLE table=test-table [id=d81697ac4e494b9dac3466c7e77837f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:10:32.564210 30511 tablet_service.cc:1467] Processing CreateTablet for tablet c05f40c4d756417b9d3acbed2b6dea62 (DEFAULT_TABLE table=test-table [id=d81697ac4e494b9dac3466c7e77837f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:10:32.565372 30374 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c05f40c4d756417b9d3acbed2b6dea62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:10:32.565385 30511 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c05f40c4d756417b9d3acbed2b6dea62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:10:32.566996 30239 tablet_service.cc:1467] Processing CreateTablet for tablet c05f40c4d756417b9d3acbed2b6dea62 (DEFAULT_TABLE table=test-table [id=d81697ac4e494b9dac3466c7e77837f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:10:32.568162 30239 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c05f40c4d756417b9d3acbed2b6dea62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:10:32.571825 30595 tablet_bootstrap.cc:492] T c05f40c4d756417b9d3acbed2b6dea62 P 5e393127ce344afd90b6d5a0677dacd2: Bootstrap starting.
I20260805 23:10:32.573681 30596 tablet_bootstrap.cc:492] T c05f40c4d756417b9d3acbed2b6dea62 P 8c9dc7929d514cfb86d62699cd53dd1b: Bootstrap starting.
I20260805 23:10:32.574508 30595 tablet_bootstrap.cc:654] T c05f40c4d756417b9d3acbed2b6dea62 P 5e393127ce344afd90b6d5a0677dacd2: Neither blocks nor log segments found. Creating new log.
I20260805 23:10:32.576052 30596 tablet_bootstrap.cc:654] T c05f40c4d756417b9d3acbed2b6dea62 P 8c9dc7929d514cfb86d62699cd53dd1b: Neither blocks nor log segments found. Creating new log.
I20260805 23:10:32.576012 30597 tablet_bootstrap.cc:492] T c05f40c4d756417b9d3acbed2b6dea62 P 694df89106e64d9abd5b7e1fe899776a: Bootstrap starting.
I20260805 23:10:32.576228 30595 log.cc:824] T c05f40c4d756417b9d3acbed2b6dea62 P 5e393127ce344afd90b6d5a0677dacd2: Log is configured to *not* fsync() on all Append() calls
I20260805 23:10:32.577482 30596 log.cc:824] T c05f40c4d756417b9d3acbed2b6dea62 P 8c9dc7929d514cfb86d62699cd53dd1b: Log is configured to *not* fsync() on all Append() calls
I20260805 23:10:32.578395 30597 tablet_bootstrap.cc:654] T c05f40c4d756417b9d3acbed2b6dea62 P 694df89106e64d9abd5b7e1fe899776a: Neither blocks nor log segments found. Creating new log.
I20260805 23:10:32.578719 30595 tablet_bootstrap.cc:492] T c05f40c4d756417b9d3acbed2b6dea62 P 5e393127ce344afd90b6d5a0677dacd2: No bootstrap required, opened a new log
I20260805 23:10:32.578934 30595 ts_tablet_manager.cc:1397] T c05f40c4d756417b9d3acbed2b6dea62 P 5e393127ce344afd90b6d5a0677dacd2: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260805 23:10:32.579228 30597 log.cc:824] T c05f40c4d756417b9d3acbed2b6dea62 P 694df89106e64d9abd5b7e1fe899776a: Log is configured to *not* fsync() on all Append() calls
I20260805 23:10:32.579691 30596 tablet_bootstrap.cc:492] T c05f40c4d756417b9d3acbed2b6dea62 P 8c9dc7929d514cfb86d62699cd53dd1b: No bootstrap required, opened a new log
I20260805 23:10:32.579887 30596 ts_tablet_manager.cc:1397] T c05f40c4d756417b9d3acbed2b6dea62 P 8c9dc7929d514cfb86d62699cd53dd1b: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.002s
I20260805 23:10:32.581276 30597 tablet_bootstrap.cc:492] T c05f40c4d756417b9d3acbed2b6dea62 P 694df89106e64d9abd5b7e1fe899776a: No bootstrap required, opened a new log
I20260805 23:10:32.581491 30597 ts_tablet_manager.cc:1397] T c05f40c4d756417b9d3acbed2b6dea62 P 694df89106e64d9abd5b7e1fe899776a: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260805 23:10:32.583616 30595 raft_consensus.cc:348] T c05f40c4d756417b9d3acbed2b6dea62 P 5e393127ce344afd90b6d5a0677dacd2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c9dc7929d514cfb86d62699cd53dd1b" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 45393 } } peers { permanent_uuid: "5e393127ce344afd90b6d5a0677dacd2" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 35309 } } peers { permanent_uuid: "694df89106e64d9abd5b7e1fe899776a" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 46277 } }
I20260805 23:10:32.583936 30595 raft_consensus.cc:374] T c05f40c4d756417b9d3acbed2b6dea62 P 5e393127ce344afd90b6d5a0677dacd2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:10:32.584010 30595 raft_consensus.cc:729] T c05f40c4d756417b9d3acbed2b6dea62 P 5e393127ce344afd90b6d5a0677dacd2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e393127ce344afd90b6d5a0677dacd2, State: Initialized, Role: FOLLOWER
I20260805 23:10:32.584131 30596 raft_consensus.cc:348] T c05f40c4d756417b9d3acbed2b6dea62 P 8c9dc7929d514cfb86d62699cd53dd1b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c9dc7929d514cfb86d62699cd53dd1b" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 45393 } } peers { permanent_uuid: "5e393127ce344afd90b6d5a0677dacd2" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 35309 } } peers { permanent_uuid: "694df89106e64d9abd5b7e1fe899776a" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 46277 } }
I20260805 23:10:32.584393 30596 raft_consensus.cc:374] T c05f40c4d756417b9d3acbed2b6dea62 P 8c9dc7929d514cfb86d62699cd53dd1b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:10:32.584450 30596 raft_consensus.cc:729] T c05f40c4d756417b9d3acbed2b6dea62 P 8c9dc7929d514cfb86d62699cd53dd1b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c9dc7929d514cfb86d62699cd53dd1b, State: Initialized, Role: FOLLOWER
I20260805 23:10:32.584537 30595 consensus_queue.cc:260] T c05f40c4d756417b9d3acbed2b6dea62 P 5e393127ce344afd90b6d5a0677dacd2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c9dc7929d514cfb86d62699cd53dd1b" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 45393 } } peers { permanent_uuid: "5e393127ce344afd90b6d5a0677dacd2" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 35309 } } peers { permanent_uuid: "694df89106e64d9abd5b7e1fe899776a" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 46277 } }
I20260805 23:10:32.584882 30596 consensus_queue.cc:260] T c05f40c4d756417b9d3acbed2b6dea62 P 8c9dc7929d514cfb86d62699cd53dd1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c9dc7929d514cfb86d62699cd53dd1b" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 45393 } } peers { permanent_uuid: "5e393127ce344afd90b6d5a0677dacd2" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 35309 } } peers { permanent_uuid: "694df89106e64d9abd5b7e1fe899776a" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 46277 } }
I20260805 23:10:32.585341 30442 heartbeater.cc:503] Master 127.25.92.254:46003 was elected leader, sending a full tablet report...
I20260805 23:10:32.585381 30597 raft_consensus.cc:348] T c05f40c4d756417b9d3acbed2b6dea62 P 694df89106e64d9abd5b7e1fe899776a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c9dc7929d514cfb86d62699cd53dd1b" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 45393 } } peers { permanent_uuid: "5e393127ce344afd90b6d5a0677dacd2" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 35309 } } peers { permanent_uuid: "694df89106e64d9abd5b7e1fe899776a" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 46277 } }
I20260805 23:10:32.585681 30597 raft_consensus.cc:374] T c05f40c4d756417b9d3acbed2b6dea62 P 694df89106e64d9abd5b7e1fe899776a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:10:32.585748 30597 raft_consensus.cc:729] T c05f40c4d756417b9d3acbed2b6dea62 P 694df89106e64d9abd5b7e1fe899776a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 694df89106e64d9abd5b7e1fe899776a, State: Initialized, Role: FOLLOWER
I20260805 23:10:32.585911 30596 ts_tablet_manager.cc:1428] T c05f40c4d756417b9d3acbed2b6dea62 P 8c9dc7929d514cfb86d62699cd53dd1b: Time spent starting tablet: real 0.006s user 0.006s sys 0.002s
I20260805 23:10:32.586174 30597 consensus_queue.cc:260] T c05f40c4d756417b9d3acbed2b6dea62 P 694df89106e64d9abd5b7e1fe899776a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c9dc7929d514cfb86d62699cd53dd1b" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 45393 } } peers { permanent_uuid: "5e393127ce344afd90b6d5a0677dacd2" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 35309 } } peers { permanent_uuid: "694df89106e64d9abd5b7e1fe899776a" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 46277 } }
I20260805 23:10:32.586262 30578 heartbeater.cc:503] Master 127.25.92.254:46003 was elected leader, sending a full tablet report...
I20260805 23:10:32.586861 30595 ts_tablet_manager.cc:1428] T c05f40c4d756417b9d3acbed2b6dea62 P 5e393127ce344afd90b6d5a0677dacd2: Time spent starting tablet: real 0.008s user 0.008s sys 0.000s
I20260805 23:10:32.587131 30597 ts_tablet_manager.cc:1428] T c05f40c4d756417b9d3acbed2b6dea62 P 694df89106e64d9abd5b7e1fe899776a: Time spent starting tablet: real 0.006s user 0.002s sys 0.003s
I20260805 23:10:32.587513 30307 heartbeater.cc:503] Master 127.25.92.254:46003 was elected leader, sending a full tablet report...
I20260805 23:10:32.737650 30601 raft_consensus.cc:482] T c05f40c4d756417b9d3acbed2b6dea62 P 5e393127ce344afd90b6d5a0677dacd2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 23:10:32.737912 30601 raft_consensus.cc:504] T c05f40c4d756417b9d3acbed2b6dea62 P 5e393127ce344afd90b6d5a0677dacd2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c9dc7929d514cfb86d62699cd53dd1b" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 45393 } } peers { permanent_uuid: "5e393127ce344afd90b6d5a0677dacd2" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 35309 } } peers { permanent_uuid: "694df89106e64d9abd5b7e1fe899776a" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 46277 } }
I20260805 23:10:32.739212 30601 leader_election.cc:288] T c05f40c4d756417b9d3acbed2b6dea62 P 5e393127ce344afd90b6d5a0677dacd2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c9dc7929d514cfb86d62699cd53dd1b (127.25.92.195:45393), 694df89106e64d9abd5b7e1fe899776a (127.25.92.193:46277)
I20260805 23:10:32.743288 30590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:32.739606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.194:59531 (local address 127.25.92.195:45393)
0805 23:10:32.739771 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:32.739777 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:32.739796 (+ 19us) server_negotiation.cc:407] Connection header received
0805 23:10:32.739998 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:32.740002 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:32.740048 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:32.740140 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:10:32.740421 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:32.740990 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:32.742212 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:32.742666 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:32.742806 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:32.742865 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:32.743067 (+ 202us) server_negotiation.cc:299] Negotiation successful
0805 23:10:32.743143 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260805 23:10:32.743288 30444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:32.739466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.195:45393 (local address 127.25.92.194:59531)
0805 23:10:32.739703 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:32.739722 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:32.739835 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:32.740155 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:32.740158 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:32.740176 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:10:32.740299 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:32.740306 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:32.741120 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:32.741125 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:32.742068 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:32.742078 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:32.742806 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:32.742810 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:32.742933 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:32.742953 (+ 20us) client_negotiation.cc:769] Sending connection context
0805 23:10:32.743057 (+ 104us) client_negotiation.cc:240] Negotiation successful
0805 23:10:32.743143 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143}
I20260805 23:10:32.743470 30593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:32.739792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.194:53841 (local address 127.25.92.193:46277)
0805 23:10:32.739926 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:32.739932 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:32.740006 (+ 74us) server_negotiation.cc:407] Connection header received
0805 23:10:32.740199 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:32.740213 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:32.740257 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:32.740344 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:10:32.740643 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:32.741180 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:32.742360 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:32.742820 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:32.742965 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:32.743022 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:32.743229 (+ 207us) server_negotiation.cc:299] Negotiation successful
0805 23:10:32.743310 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260805 23:10:32.743486 30604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:32.739562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.193:46277 (local address 127.25.92.194:53841)
0805 23:10:32.739924 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:32.739940 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:32.740075 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:32.740371 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:32.740374 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:32.740390 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:10:32.740526 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:32.740532 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:32.741301 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:32.741307 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:32.742225 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:32.742234 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:32.742965 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:32.742969 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:32.743092 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:32.743120 (+ 28us) client_negotiation.cc:769] Sending connection context
0805 23:10:32.743211 (+ 91us) client_negotiation.cc:240] Negotiation successful
0805 23:10:32.743287 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":115,"threads_started":1}
I20260805 23:10:32.744143 30531 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c05f40c4d756417b9d3acbed2b6dea62" candidate_uuid: "5e393127ce344afd90b6d5a0677dacd2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c9dc7929d514cfb86d62699cd53dd1b" is_pre_election: true
I20260805 23:10:32.744225 30260 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c05f40c4d756417b9d3acbed2b6dea62" candidate_uuid: "5e393127ce344afd90b6d5a0677dacd2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "694df89106e64d9abd5b7e1fe899776a" is_pre_election: true
I20260805 23:10:32.744524 30531 raft_consensus.cc:2454] T c05f40c4d756417b9d3acbed2b6dea62 P 8c9dc7929d514cfb86d62699cd53dd1b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e393127ce344afd90b6d5a0677dacd2 in term 0.
I20260805 23:10:32.744570 30260 raft_consensus.cc:2454] T c05f40c4d756417b9d3acbed2b6dea62 P 694df89106e64d9abd5b7e1fe899776a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e393127ce344afd90b6d5a0677dacd2 in term 0.
I20260805 23:10:32.745061 30327 leader_election.cc:302] T c05f40c4d756417b9d3acbed2b6dea62 P 5e393127ce344afd90b6d5a0677dacd2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e393127ce344afd90b6d5a0677dacd2, 694df89106e64d9abd5b7e1fe899776a; no voters:
I20260805 23:10:32.745383 30601 raft_consensus.cc:2781] T c05f40c4d756417b9d3acbed2b6dea62 P 5e393127ce344afd90b6d5a0677dacd2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260805 23:10:32.745504 30601 raft_consensus.cc:482] T c05f40c4d756417b9d3acbed2b6dea62 P 5e393127ce344afd90b6d5a0677dacd2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260805 23:10:32.745586 30601 raft_consensus.cc:3037] T c05f40c4d756417b9d3acbed2b6dea62 P 5e393127ce344afd90b6d5a0677dacd2 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:10:32.746753 30601 raft_consensus.cc:504] T c05f40c4d756417b9d3acbed2b6dea62 P 5e393127ce344afd90b6d5a0677dacd2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c9dc7929d514cfb86d62699cd53dd1b" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 45393 } } peers { permanent_uuid: "5e393127ce344afd90b6d5a0677dacd2" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 35309 } } peers { permanent_uuid: "694df89106e64d9abd5b7e1fe899776a" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 46277 } }
I20260805 23:10:32.747232 30601 leader_election.cc:288] T c05f40c4d756417b9d3acbed2b6dea62 P 5e393127ce344afd90b6d5a0677dacd2 [CANDIDATE]: Term 1 election: Requested vote from peers 8c9dc7929d514cfb86d62699cd53dd1b (127.25.92.195:45393), 694df89106e64d9abd5b7e1fe899776a (127.25.92.193:46277)
I20260805 23:10:32.747665 30260 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c05f40c4d756417b9d3acbed2b6dea62" candidate_uuid: "5e393127ce344afd90b6d5a0677dacd2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "694df89106e64d9abd5b7e1fe899776a"
I20260805 23:10:32.747684 30531 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c05f40c4d756417b9d3acbed2b6dea62" candidate_uuid: "5e393127ce344afd90b6d5a0677dacd2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c9dc7929d514cfb86d62699cd53dd1b"
I20260805 23:10:32.747802 30531 raft_consensus.cc:3037] T c05f40c4d756417b9d3acbed2b6dea62 P 8c9dc7929d514cfb86d62699cd53dd1b [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:10:32.747817 30260 raft_consensus.cc:3037] T c05f40c4d756417b9d3acbed2b6dea62 P 694df89106e64d9abd5b7e1fe899776a [term 0 FOLLOWER]: Advancing to term 1
W20260805 23:10:32.748865 30579 tablet.cc:2369] T c05f40c4d756417b9d3acbed2b6dea62 P 8c9dc7929d514cfb86d62699cd53dd1b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260805 23:10:32.748984 30531 raft_consensus.cc:2454] T c05f40c4d756417b9d3acbed2b6dea62 P 8c9dc7929d514cfb86d62699cd53dd1b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e393127ce344afd90b6d5a0677dacd2 in term 1.
I20260805 23:10:32.749166 30260 raft_consensus.cc:2454] T c05f40c4d756417b9d3acbed2b6dea62 P 694df89106e64d9abd5b7e1fe899776a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e393127ce344afd90b6d5a0677dacd2 in term 1.
I20260805 23:10:32.749383 30329 leader_election.cc:302] T c05f40c4d756417b9d3acbed2b6dea62 P 5e393127ce344afd90b6d5a0677dacd2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e393127ce344afd90b6d5a0677dacd2, 8c9dc7929d514cfb86d62699cd53dd1b; no voters:
I20260805 23:10:32.749610 30601 raft_consensus.cc:2781] T c05f40c4d756417b9d3acbed2b6dea62 P 5e393127ce344afd90b6d5a0677dacd2 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:10:32.749840 30601 raft_consensus.cc:686] T c05f40c4d756417b9d3acbed2b6dea62 P 5e393127ce344afd90b6d5a0677dacd2 [term 1 LEADER]: Becoming Leader. State: Replica: 5e393127ce344afd90b6d5a0677dacd2, State: Running, Role: LEADER
I20260805 23:10:32.750191 30601 consensus_queue.cc:237] T c05f40c4d756417b9d3acbed2b6dea62 P 5e393127ce344afd90b6d5a0677dacd2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c9dc7929d514cfb86d62699cd53dd1b" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 45393 } } peers { permanent_uuid: "5e393127ce344afd90b6d5a0677dacd2" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 35309 } } peers { permanent_uuid: "694df89106e64d9abd5b7e1fe899776a" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 46277 } }
I20260805 23:10:32.753767 30114 catalog_manager.cc:5495] T c05f40c4d756417b9d3acbed2b6dea62 P 5e393127ce344afd90b6d5a0677dacd2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e393127ce344afd90b6d5a0677dacd2 (127.25.92.194). New cstate: current_term: 1 leader_uuid: "5e393127ce344afd90b6d5a0677dacd2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c9dc7929d514cfb86d62699cd53dd1b" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 45393 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e393127ce344afd90b6d5a0677dacd2" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 35309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "694df89106e64d9abd5b7e1fe899776a" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 46277 } health_report { overall_health: UNKNOWN } } }
W20260805 23:10:32.774832 30443 tablet.cc:2369] T c05f40c4d756417b9d3acbed2b6dea62 P 5e393127ce344afd90b6d5a0677dacd2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260805 23:10:32.776577 30308 tablet.cc:2369] T c05f40c4d756417b9d3acbed2b6dea62 P 694df89106e64d9abd5b7e1fe899776a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260805 23:10:32.801548 30594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:32.798035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42576 (local address 127.25.92.194:35309)
0805 23:10:32.798159 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:32.798162 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:32.798176 (+ 14us) server_negotiation.cc:407] Connection header received
0805 23:10:32.798281 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:32.798283 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:32.798338 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:32.798432 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 23:10:32.798665 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:32.799165 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:32.799966 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:32.800179 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:32.800294 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:32.800621 (+ 327us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 23:10:32.801076 (+ 455us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 23:10:32.801206 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:32.801265 (+ 59us) server_negotiation.cc:299] Negotiation successful
0805 23:10:32.801323 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260805 23:10:32.807514 30260 raft_consensus.cc:1261] T c05f40c4d756417b9d3acbed2b6dea62 P 694df89106e64d9abd5b7e1fe899776a [term 1 FOLLOWER]: Refusing update from remote peer 5e393127ce344afd90b6d5a0677dacd2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:10:32.807523 30531 raft_consensus.cc:1261] T c05f40c4d756417b9d3acbed2b6dea62 P 8c9dc7929d514cfb86d62699cd53dd1b [term 1 FOLLOWER]: Refusing update from remote peer 5e393127ce344afd90b6d5a0677dacd2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:10:32.808348 30605 consensus_queue.cc:1035] T c05f40c4d756417b9d3acbed2b6dea62 P 5e393127ce344afd90b6d5a0677dacd2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "694df89106e64d9abd5b7e1fe899776a" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:10:32.808555 30601 consensus_queue.cc:1035] T c05f40c4d756417b9d3acbed2b6dea62 P 5e393127ce344afd90b6d5a0677dacd2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c9dc7929d514cfb86d62699cd53dd1b" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 45393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:10:32.817997 30609 mvcc.cc:205] Tried to move back new op lower bound from 7315338988771790848 to 7315338988547952640. Current Snapshot: MvccSnapshot[applied={T|T < 7315338988771790848}]
I20260805 23:10:32.819463 30608 mvcc.cc:205] Tried to move back new op lower bound from 7315338988771790848 to 7315338988547952640. Current Snapshot: MvccSnapshot[applied={T|T < 7315338988771790848 or (T in {7315338988771790848})}]
I20260805 23:10:32.824972 30114 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37236:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:10:32.825158 30114 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37236:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:10:32.827885 30114 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e358b140f5304d1d9b5356a045bd1635: Sending DeleteTablet for 3 replicas of tablet c05f40c4d756417b9d3acbed2b6dea62
I20260805 23:10:32.828683 30374 tablet_service.cc:1514] Processing DeleteTablet for tablet c05f40c4d756417b9d3acbed2b6dea62 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:10:32 UTC) from {username='kudu', principal='kudu/127.25.92.254@KRBTEST.COM'} at 127.0.0.1:42564
I20260805 23:10:32.828807 30239 tablet_service.cc:1514] Processing DeleteTablet for tablet c05f40c4d756417b9d3acbed2b6dea62 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:10:32 UTC) from {username='kudu', principal='kudu/127.25.92.254@KRBTEST.COM'} at 127.0.0.1:50602
I20260805 23:10:32.828821 30511 tablet_service.cc:1514] Processing DeleteTablet for tablet c05f40c4d756417b9d3acbed2b6dea62 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:10:32 UTC) from {username='kudu', principal='kudu/127.25.92.254@KRBTEST.COM'} at 127.0.0.1:60490
I20260805 23:10:32.829368 30619 tablet_replica.cc:330] T c05f40c4d756417b9d3acbed2b6dea62 P 694df89106e64d9abd5b7e1fe899776a: stopping tablet replica
I20260805 23:10:32.829741 30619 raft_consensus.cc:2229] T c05f40c4d756417b9d3acbed2b6dea62 P 694df89106e64d9abd5b7e1fe899776a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 23:10:32.829800 30620 tablet_replica.cc:330] T c05f40c4d756417b9d3acbed2b6dea62 P 8c9dc7929d514cfb86d62699cd53dd1b: stopping tablet replica
I20260805 23:10:32.830014 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 30177
I20260805 23:10:32.834537 30620 raft_consensus.cc:2229] T c05f40c4d756417b9d3acbed2b6dea62 P 8c9dc7929d514cfb86d62699cd53dd1b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 23:10:32.834972 30620 raft_consensus.cc:2258] T c05f40c4d756417b9d3acbed2b6dea62 P 8c9dc7929d514cfb86d62699cd53dd1b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:10:32.835424 30618 tablet_replica.cc:330] T c05f40c4d756417b9d3acbed2b6dea62 P 5e393127ce344afd90b6d5a0677dacd2: stopping tablet replica
I20260805 23:10:32.835769 30618 raft_consensus.cc:2229] T c05f40c4d756417b9d3acbed2b6dea62 P 5e393127ce344afd90b6d5a0677dacd2 [term 1 LEADER]: Raft consensus shutting down.
I20260805 23:10:32.836162 30618 raft_consensus.cc:2258] T c05f40c4d756417b9d3acbed2b6dea62 P 5e393127ce344afd90b6d5a0677dacd2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:10:32.837594 30620 ts_tablet_manager.cc:1905] T c05f40c4d756417b9d3acbed2b6dea62 P 8c9dc7929d514cfb86d62699cd53dd1b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:10:32.837594 30618 ts_tablet_manager.cc:1905] T c05f40c4d756417b9d3acbed2b6dea62 P 5e393127ce344afd90b6d5a0677dacd2: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260805 23:10:32.840453 30098 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.25.92.193:46277 (error 108)
I20260805 23:10:32.840695 30618 ts_tablet_manager.cc:1918] T c05f40c4d756417b9d3acbed2b6dea62 P 5e393127ce344afd90b6d5a0677dacd2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 23:10:32.840732 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 30312
I20260805 23:10:32.840759 30620 ts_tablet_manager.cc:1918] T c05f40c4d756417b9d3acbed2b6dea62 P 8c9dc7929d514cfb86d62699cd53dd1b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 23:10:32.841913 30620 log.cc:1196] T c05f40c4d756417b9d3acbed2b6dea62 P 8c9dc7929d514cfb86d62699cd53dd1b: Deleting WAL directory at /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971406251336-25971-0/minicluster-data/ts-2/wal/wals/c05f40c4d756417b9d3acbed2b6dea62
I20260805 23:10:32.842270 30620 ts_tablet_manager.cc:1939] T c05f40c4d756417b9d3acbed2b6dea62 P 8c9dc7929d514cfb86d62699cd53dd1b: Deleting consensus metadata
I20260805 23:10:32.843663 30100 catalog_manager.cc:4841] TS 8c9dc7929d514cfb86d62699cd53dd1b (127.25.92.195:45393): tablet c05f40c4d756417b9d3acbed2b6dea62 (table test-table [id=d81697ac4e494b9dac3466c7e77837f3]) successfully deleted
I20260805 23:10:32.846602 30592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:32.846287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.193:46277 (local address 127.0.0.1:50610)
0805 23:10:32.846436 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:32.846508 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.193:46277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260805 23:10:32.846907 30098 catalog_manager.cc:4584] TS 694df89106e64d9abd5b7e1fe899776a (127.25.92.193:46277): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c05f40c4d756417b9d3acbed2b6dea62: Network error: Client connection negotiation failed: client connection to 127.25.92.193:46277: connect: Connection refused (error 111)
I20260805 23:10:32.850941 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 30447
I20260805 23:10:32.852232 30592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:32.851189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.194:35309 (local address 127.0.0.1:42590)
0805 23:10:32.852033 (+ 844us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:32.852110 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.194:35309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":730}
I20260805 23:10:32.857434 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 30083
2026-08-05T23:10:32Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2541 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-05T23:10:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:10:32Z Disabled control of system clock
I20260805 23:10:32.889768 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971406251336-25971-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971406251336-25971-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971406251336-25971-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.254:40757
--webserver_interface=127.25.92.254
--webserver_port=0
--builtin_ntp_servers=127.25.92.212:33915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.92.254:40757
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260805 23:10:33.003265 30626 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:10:33.003559 30626 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:10:33.003607 30626 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:10:33.007741 30626 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:10:33.007809 30626 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:10:33.007825 30626 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:10:33.007843 30626 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:10:33.007858 30626 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:10:33.012801 30626 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:33915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971406251336-25971-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971406251336-25971-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.92.254:40757
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.254:40757
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.92.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30626
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971406251336-25971-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:10:33.013953 30626 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:10:33.015388 30626 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:10:33.024305 30631 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:33.024356 30634 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:10:33.024356 30626 server_base.cc:1034] running on GCE node
W20260805 23:10:33.024541 30632 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:10:33.025174 30626 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:10:33.026346 30626 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:10:33.027522 30626 hybrid_clock.cc:648] HybridClock initialized: now 1785971433027489 us; error 45 us; skew 500 ppm
I20260805 23:10:33.029611 30626 webserver.cc:460] Webserver started at http://127.25.92.254:44983/ using document root <none> and password file <none>
I20260805 23:10:33.030251 30626 fs_manager.cc:359] Metadata directory not provided
I20260805 23:10:33.030318 30626 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:10:33.030572 30626 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:10:33.032928 30626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971406251336-25971-0/minicluster-data/master-0/data/instance:
uuid: "6af8d513de3142a5be8057f17a7dd6b4"
format_stamp: "Formatted at 2026-08-05 23:10:33 on dist-test-slave-ss4n"
server_key: "8db322fa2d9a3c21bc5d32366d61d8c5"
server_key_iv: "8fbca92f2e8c73b578162e58d2254e3a"
server_key_version: "encryptionkey@0"
I20260805 23:10:33.033708 30626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971406251336-25971-0/minicluster-data/master-0/wal/instance:
uuid: "6af8d513de3142a5be8057f17a7dd6b4"
format_stamp: "Formatted at 2026-08-05 23:10:33 on dist-test-slave-ss4n"
server_key: "8db322fa2d9a3c21bc5d32366d61d8c5"
server_key_iv: "8fbca92f2e8c73b578162e58d2254e3a"
server_key_version: "encryptionkey@0"
I20260805 23:10:33.038015 30626 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260805 23:10:33.040747 30640 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:33.042237 30626 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:10:33.042397 30626 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971406251336-25971-0/minicluster-data/master-0/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971406251336-25971-0/minicluster-data/master-0/wal
uuid: "6af8d513de3142a5be8057f17a7dd6b4"
format_stamp: "Formatted at 2026-08-05 23:10:33 on dist-test-slave-ss4n"
server_key: "8db322fa2d9a3c21bc5d32366d61d8c5"
server_key_iv: "8fbca92f2e8c73b578162e58d2254e3a"
server_key_version: "encryptionkey@0"
I20260805 23:10:33.042563 30626 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971406251336-25971-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971406251336-25971-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971406251336-25971-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:10:33.048044 30626 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:10:33.048784 30626 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:10:33.049000 30626 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:10:33.057242 30626 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.254:40757
I20260805 23:10:33.057255 30692 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.254:40757 every 8 connection(s)
I20260805 23:10:33.058349 30626 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
I20260805 23:10:33.061856 30693 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:10:33.065958 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 30626
I20260805 23:10:33.066135 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971406251336-25971-0/minicluster-data/master-0/wal/instance
I20260805 23:10:33.066431 25971 external_mini_cluster.cc:1428] Setting key a79908d007b0160b9677181c474bf2ef
I20260805 23:10:33.067971 30693 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6af8d513de3142a5be8057f17a7dd6b4: Bootstrap starting.
I20260805 23:10:33.070374 30693 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6af8d513de3142a5be8057f17a7dd6b4: Neither blocks nor log segments found. Creating new log.
I20260805 23:10:33.071260 30693 log.cc:824] T 00000000000000000000000000000000 P 6af8d513de3142a5be8057f17a7dd6b4: Log is configured to *not* fsync() on all Append() calls
I20260805 23:10:33.072564 30696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:33.067985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54376 (local address 127.25.92.254:40757)
0805 23:10:33.068739 (+ 754us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:33.068756 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:33.068795 (+ 39us) server_negotiation.cc:407] Connection header received
0805 23:10:33.069471 (+ 676us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:33.069549 (+ 78us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:33.069901 (+ 352us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:33.070260 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:33.071197 (+ 937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:33.071202 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:33.071213 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:10:33.071223 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:33.071279 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:10:33.071295 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:33.071410 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:33.071517 (+ 107us) server_negotiation.cc:299] Negotiation successful
0805 23:10:33.071769 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":174,"threads_started":1}
I20260805 23:10:33.073330 30693 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6af8d513de3142a5be8057f17a7dd6b4: No bootstrap required, opened a new log
I20260805 23:10:33.076970 30693 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6af8d513de3142a5be8057f17a7dd6b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6af8d513de3142a5be8057f17a7dd6b4" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 40757 } }
I20260805 23:10:33.077235 30693 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6af8d513de3142a5be8057f17a7dd6b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:10:33.077320 30693 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6af8d513de3142a5be8057f17a7dd6b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6af8d513de3142a5be8057f17a7dd6b4, State: Initialized, Role: FOLLOWER
I20260805 23:10:33.077796 30693 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6af8d513de3142a5be8057f17a7dd6b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6af8d513de3142a5be8057f17a7dd6b4" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 40757 } }
I20260805 23:10:33.077945 30693 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6af8d513de3142a5be8057f17a7dd6b4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:10:33.078028 30693 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6af8d513de3142a5be8057f17a7dd6b4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:10:33.078130 30693 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6af8d513de3142a5be8057f17a7dd6b4 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:10:33.079177 30693 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6af8d513de3142a5be8057f17a7dd6b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6af8d513de3142a5be8057f17a7dd6b4" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 40757 } }
I20260805 23:10:33.079517 30693 leader_election.cc:302] T 00000000000000000000000000000000 P 6af8d513de3142a5be8057f17a7dd6b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6af8d513de3142a5be8057f17a7dd6b4; no voters:
I20260805 23:10:33.079846 30693 leader_election.cc:288] T 00000000000000000000000000000000 P 6af8d513de3142a5be8057f17a7dd6b4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 23:10:33.079986 30698 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6af8d513de3142a5be8057f17a7dd6b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:10:33.080255 30698 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6af8d513de3142a5be8057f17a7dd6b4 [term 1 LEADER]: Becoming Leader. State: Replica: 6af8d513de3142a5be8057f17a7dd6b4, State: Running, Role: LEADER
I20260805 23:10:33.080557 30698 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6af8d513de3142a5be8057f17a7dd6b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6af8d513de3142a5be8057f17a7dd6b4" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 40757 } }
I20260805 23:10:33.081202 30693 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6af8d513de3142a5be8057f17a7dd6b4 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:10:33.082368 30700 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6af8d513de3142a5be8057f17a7dd6b4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6af8d513de3142a5be8057f17a7dd6b4. Latest consensus state: current_term: 1 leader_uuid: "6af8d513de3142a5be8057f17a7dd6b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6af8d513de3142a5be8057f17a7dd6b4" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 40757 } } }
I20260805 23:10:33.082388 30699 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6af8d513de3142a5be8057f17a7dd6b4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6af8d513de3142a5be8057f17a7dd6b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6af8d513de3142a5be8057f17a7dd6b4" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 40757 } } }
I20260805 23:10:33.082546 30700 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6af8d513de3142a5be8057f17a7dd6b4 [sys.catalog]: This master's current role is: LEADER
I20260805 23:10:33.082549 30699 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6af8d513de3142a5be8057f17a7dd6b4 [sys.catalog]: This master's current role is: LEADER
I20260805 23:10:33.082901 30709 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:10:33.086062 30709 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:10:33.091827 30709 catalog_manager.cc:1347] Generated new cluster ID: bd15f433aa68447db6b126aa2748207a
I20260805 23:10:33.091912 30709 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:10:33.107642 30709 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:10:33.107993 30709 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:10:33.113916 30709 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6af8d513de3142a5be8057f17a7dd6b4: Generated new TSK 0
I20260805 23:10:33.114748 30709 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260805 23:10:33.129050 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971406251336-25971-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971406251336-25971-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971406251336-25971-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971406251336-25971-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.92.193:0
--local_ip_for_outbound_sockets=127.25.92.193
--webserver_interface=127.25.92.193
--webserver_port=0
--tserver_master_addrs=127.25.92.254:40757
--builtin_ntp_servers=127.25.92.212:33915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260805 23:10:33.245421 30719 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:10:33.245738 30719 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:10:33.245834 30719 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:10:33.250178 30719 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:10:33.250254 30719 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:10:33.250360 30719 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.92.193
I20260805 23:10:33.255322 30719 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:33915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971406251336-25971-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971406251336-25971-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.92.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971406251336-25971-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.92.193
--webserver_port=0
--tserver_master_addrs=127.25.92.254:40757
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30719
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.92.193
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971406251336-25971-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:10:33.256489 30719 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:10:33.257938 30719 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:10:33.267637 30727 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:33.267665 30724 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:10:33.267655 30719 server_base.cc:1034] running on GCE node
W20260805 23:10:33.267643 30725 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:10:33.268322 30719 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:10:33.268930 30719 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:10:33.270151 30719 hybrid_clock.cc:648] HybridClock initialized: now 1785971433270126 us; error 40 us; skew 500 ppm
I20260805 23:10:33.272298 30719 webserver.cc:460] Webserver started at http://127.25.92.193:45007/ using document root <none> and password file <none>
I20260805 23:10:33.272953 30719 fs_manager.cc:359] Metadata directory not provided
I20260805 23:10:33.273054 30719 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:10:33.273303 30719 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:10:33.275678 30719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971406251336-25971-0/minicluster-data/ts-0/data/instance:
uuid: "80847a8a70584d3caab29edd59bed44a"
format_stamp: "Formatted at 2026-08-05 23:10:33 on dist-test-slave-ss4n"
server_key: "81e19789d7f850edd7f7d7a5d71b7e05"
server_key_iv: "3cbd4d1a0b970c4694b61a17d7b33f28"
server_key_version: "encryptionkey@0"
I20260805 23:10:33.276319 30719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971406251336-25971-0/minicluster-data/ts-0/wal/instance:
uuid: "80847a8a70584d3caab29edd59bed44a"
format_stamp: "Formatted at 2026-08-05 23:10:33 on dist-test-slave-ss4n"
server_key: "81e19789d7f850edd7f7d7a5d71b7e05"
server_key_iv: "3cbd4d1a0b970c4694b61a17d7b33f28"
server_key_version: "encryptionkey@0"
I20260805 23:10:33.280577 30719 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260805 23:10:33.283524 30733 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:33.284878 30719 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260805 23:10:33.285024 30719 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971406251336-25971-0/minicluster-data/ts-0/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971406251336-25971-0/minicluster-data/ts-0/wal
uuid: "80847a8a70584d3caab29edd59bed44a"
format_stamp: "Formatted at 2026-08-05 23:10:33 on dist-test-slave-ss4n"
server_key: "81e19789d7f850edd7f7d7a5d71b7e05"
server_key_iv: "3cbd4d1a0b970c4694b61a17d7b33f28"
server_key_version: "encryptionkey@0"
I20260805 23:10:33.285162 30719 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971406251336-25971-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971406251336-25971-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971406251336-25971-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:10:33.290395 30719 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:10:33.291067 30719 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:10:33.291270 30719 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:10:33.291894 30719 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:10:33.292883 30719 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:10:33.292966 30719 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:33.293042 30719 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:10:33.293074 30719 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:33.303285 30719 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.193:37877
I20260805 23:10:33.303308 30847 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.193:37877 every 8 connection(s)
I20260805 23:10:33.304345 30719 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971406251336-25971-0/minicluster-data/ts-0/data/info.pb
I20260805 23:10:33.305832 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 30719
I20260805 23:10:33.305963 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971406251336-25971-0/minicluster-data/ts-0/wal/instance
I20260805 23:10:33.306332 25971 external_mini_cluster.cc:1428] Setting key abcbbda3fdd27ac7fdddfd8ffd31542f
I20260805 23:10:33.308364 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971406251336-25971-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971406251336-25971-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971406251336-25971-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971406251336-25971-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.92.194:0
--local_ip_for_outbound_sockets=127.25.92.194
--webserver_interface=127.25.92.194
--webserver_port=0
--tserver_master_addrs=127.25.92.254:40757
--builtin_ntp_servers=127.25.92.212:33915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260805 23:10:33.311513 30696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:33.306354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.193:45407 (local address 127.25.92.254:40757)
0805 23:10:33.308029 (+ 1675us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:33.308034 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:33.308051 (+ 17us) server_negotiation.cc:407] Connection header received
0805 23:10:33.309631 (+ 1580us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:33.309635 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:33.309681 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:33.309769 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:33.310988 (+ 1219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:33.310991 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:33.310995 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:10:33.311009 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:33.311033 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:10:33.311049 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:33.311152 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:33.311316 (+ 164us) server_negotiation.cc:299] Negotiation successful
0805 23:10:33.311367 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1585}
I20260805 23:10:33.312484 30850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:33.306981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:40757 (local address 127.25.92.193:45407)
0805 23:10:33.307780 (+ 799us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:33.307837 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:33.309378 (+ 1541us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:33.309940 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:33.309947 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:33.310048 (+ 101us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:33.310709 (+ 661us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0805 23:10:33.310711 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:33.310754 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0805 23:10:33.310827 (+ 73us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:10:33.311147 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:10:33.311157 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:10:33.311166 (+ 9us) client_negotiation.cc:769] Sending connection context
0805 23:10:33.311358 (+ 192us) client_negotiation.cc:240] Negotiation successful
0805 23:10:33.311620 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":164,"threads_started":1}
I20260805 23:10:33.313951 30848 heartbeater.cc:347] Connected to a master server at 127.25.92.254:40757
I20260805 23:10:33.314216 30848 heartbeater.cc:464] Registering TS with master...
I20260805 23:10:33.314639 30848 heartbeater.cc:511] Master 127.25.92.254:40757 requested a full tablet report, sending...
I20260805 23:10:33.316226 30657 ts_manager.cc:194] Registered new tserver with Master: 80847a8a70584d3caab29edd59bed44a (127.25.92.193:37877)
W20260805 23:10:33.422906 30851 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:10:33.423236 30851 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:10:33.423312 30851 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:10:33.427433 30851 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:10:33.427531 30851 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:10:33.427629 30851 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.92.194
I20260805 23:10:33.432626 30851 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:33915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971406251336-25971-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971406251336-25971-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.92.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971406251336-25971-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.92.194
--webserver_port=0
--tserver_master_addrs=127.25.92.254:40757
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30851
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.92.194
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971406251336-25971-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:10:33.433804 30851 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:10:33.435206 30851 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:10:33.445416 30857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:10:33.445604 30851 server_base.cc:1034] running on GCE node
W20260805 23:10:33.445429 30856 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:33.445451 30860 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:10:33.446316 30851 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:10:33.447101 30851 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:10:33.448284 30851 hybrid_clock.cc:648] HybridClock initialized: now 1785971433448248 us; error 56 us; skew 500 ppm
I20260805 23:10:33.450364 30851 webserver.cc:460] Webserver started at http://127.25.92.194:44389/ using document root <none> and password file <none>
I20260805 23:10:33.450973 30851 fs_manager.cc:359] Metadata directory not provided
I20260805 23:10:33.451068 30851 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:10:33.451299 30851 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:10:33.453442 30851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971406251336-25971-0/minicluster-data/ts-1/data/instance:
uuid: "85f1a8225a5f49a4a222cd567c3d6d66"
format_stamp: "Formatted at 2026-08-05 23:10:33 on dist-test-slave-ss4n"
server_key: "ab9222b0a93b5385a148bae8b47ec794"
server_key_iv: "1d11f7f8cce0d6b071b176fd10fc9d6b"
server_key_version: "encryptionkey@0"
I20260805 23:10:33.454085 30851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971406251336-25971-0/minicluster-data/ts-1/wal/instance:
uuid: "85f1a8225a5f49a4a222cd567c3d6d66"
format_stamp: "Formatted at 2026-08-05 23:10:33 on dist-test-slave-ss4n"
server_key: "ab9222b0a93b5385a148bae8b47ec794"
server_key_iv: "1d11f7f8cce0d6b071b176fd10fc9d6b"
server_key_version: "encryptionkey@0"
I20260805 23:10:33.457980 30851 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260805 23:10:33.460762 30866 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:33.462066 30851 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:10:33.462216 30851 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971406251336-25971-0/minicluster-data/ts-1/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971406251336-25971-0/minicluster-data/ts-1/wal
uuid: "85f1a8225a5f49a4a222cd567c3d6d66"
format_stamp: "Formatted at 2026-08-05 23:10:33 on dist-test-slave-ss4n"
server_key: "ab9222b0a93b5385a148bae8b47ec794"
server_key_iv: "1d11f7f8cce0d6b071b176fd10fc9d6b"
server_key_version: "encryptionkey@0"
I20260805 23:10:33.462338 30851 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971406251336-25971-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971406251336-25971-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971406251336-25971-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:10:33.467216 30851 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:10:33.467845 30851 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:10:33.468024 30851 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:10:33.468869 30851 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:10:33.470084 30851 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:10:33.470170 30851 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:33.470250 30851 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:10:33.470312 30851 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:33.481115 30851 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.194:39615
I20260805 23:10:33.481142 30983 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.194:39615 every 8 connection(s)
I20260805 23:10:33.482460 30851 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971406251336-25971-0/minicluster-data/ts-1/data/info.pb
I20260805 23:10:33.484968 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 30851
I20260805 23:10:33.485105 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971406251336-25971-0/minicluster-data/ts-1/wal/instance
I20260805 23:10:33.485529 25971 external_mini_cluster.cc:1428] Setting key 81b8089a831179af8b6290c29e54edbe
I20260805 23:10:33.487912 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971406251336-25971-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971406251336-25971-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971406251336-25971-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971406251336-25971-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.92.195:0
--local_ip_for_outbound_sockets=127.25.92.195
--webserver_interface=127.25.92.195
--webserver_port=0
--tserver_master_addrs=127.25.92.254:40757
--builtin_ntp_servers=127.25.92.212:33915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260805 23:10:33.489709 30696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:33.484479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.194:47111 (local address 127.25.92.254:40757)
0805 23:10:33.484655 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:33.484659 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:33.486139 (+ 1480us) server_negotiation.cc:407] Connection header received
0805 23:10:33.487682 (+ 1543us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:33.487689 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:33.487740 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:33.487854 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:33.489136 (+ 1282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:33.489144 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:33.489150 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:10:33.489169 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:33.489199 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:10:33.489214 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:33.489313 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:33.489435 (+ 122us) server_negotiation.cc:299] Negotiation successful
0805 23:10:33.489510 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260805 23:10:33.490600 30986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:33.485102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:40757 (local address 127.25.92.194:47111)
0805 23:10:33.485969 (+ 867us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:33.486028 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:33.487456 (+ 1428us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:33.488013 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:33.488024 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:33.488157 (+ 133us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:33.488886 (+ 729us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0805 23:10:33.488888 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:33.488929 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0805 23:10:33.488998 (+ 69us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:10:33.489312 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:10:33.489318 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:10:33.489322 (+ 4us) client_negotiation.cc:769] Sending connection context
0805 23:10:33.489448 (+ 126us) client_negotiation.cc:240] Negotiation successful
0805 23:10:33.489749 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":174,"threads_started":1}
I20260805 23:10:33.491894 30984 heartbeater.cc:347] Connected to a master server at 127.25.92.254:40757
I20260805 23:10:33.492128 30984 heartbeater.cc:464] Registering TS with master...
I20260805 23:10:33.492590 30984 heartbeater.cc:511] Master 127.25.92.254:40757 requested a full tablet report, sending...
I20260805 23:10:33.493758 30657 ts_manager.cc:194] Registered new tserver with Master: 85f1a8225a5f49a4a222cd567c3d6d66 (127.25.92.194:39615)
W20260805 23:10:33.606662 30987 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:10:33.606963 30987 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:10:33.607007 30987 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:10:33.611172 30987 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:10:33.611251 30987 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:10:33.611312 30987 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.92.195
I20260805 23:10:33.616364 30987 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:33915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971406251336-25971-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971406251336-25971-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.92.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971406251336-25971-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.92.195
--webserver_port=0
--tserver_master_addrs=127.25.92.254:40757
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30987
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.92.195
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971406251336-25971-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:10:33.617465 30987 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:10:33.618815 30987 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:10:33.628818 30995 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:33.628906 30992 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:33.628810 30993 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:10:33.629091 30987 server_base.cc:1034] running on GCE node
I20260805 23:10:33.629657 30987 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:10:33.630373 30987 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:10:33.631654 30987 hybrid_clock.cc:648] HybridClock initialized: now 1785971433631640 us; error 42 us; skew 500 ppm
I20260805 23:10:33.633800 30987 webserver.cc:460] Webserver started at http://127.25.92.195:37335/ using document root <none> and password file <none>
I20260805 23:10:33.634433 30987 fs_manager.cc:359] Metadata directory not provided
I20260805 23:10:33.634598 30987 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:10:33.634851 30987 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:10:33.637097 30987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971406251336-25971-0/minicluster-data/ts-2/data/instance:
uuid: "664855d22a0a4ce3b49aef4bc3332225"
format_stamp: "Formatted at 2026-08-05 23:10:33 on dist-test-slave-ss4n"
server_key: "7016dc585a737cbfc6a03ab4ea342cd8"
server_key_iv: "0b1d2e85d59e8f3328296d587d4b6e5b"
server_key_version: "encryptionkey@0"
I20260805 23:10:33.637778 30987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971406251336-25971-0/minicluster-data/ts-2/wal/instance:
uuid: "664855d22a0a4ce3b49aef4bc3332225"
format_stamp: "Formatted at 2026-08-05 23:10:33 on dist-test-slave-ss4n"
server_key: "7016dc585a737cbfc6a03ab4ea342cd8"
server_key_iv: "0b1d2e85d59e8f3328296d587d4b6e5b"
server_key_version: "encryptionkey@0"
I20260805 23:10:33.641992 30987 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260805 23:10:33.644922 31001 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:33.646325 30987 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:10:33.646478 30987 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971406251336-25971-0/minicluster-data/ts-2/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971406251336-25971-0/minicluster-data/ts-2/wal
uuid: "664855d22a0a4ce3b49aef4bc3332225"
format_stamp: "Formatted at 2026-08-05 23:10:33 on dist-test-slave-ss4n"
server_key: "7016dc585a737cbfc6a03ab4ea342cd8"
server_key_iv: "0b1d2e85d59e8f3328296d587d4b6e5b"
server_key_version: "encryptionkey@0"
I20260805 23:10:33.646610 30987 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971406251336-25971-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971406251336-25971-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971406251336-25971-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:10:33.651508 30987 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:10:33.652179 30987 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:10:33.652380 30987 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:10:33.652978 30987 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:10:33.654047 30987 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:10:33.654119 30987 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:33.654193 30987 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:10:33.654243 30987 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:33.663879 30987 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.195:34903
I20260805 23:10:33.663908 31119 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.195:34903 every 8 connection(s)
I20260805 23:10:33.664848 30987 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971406251336-25971-0/minicluster-data/ts-2/data/info.pb
I20260805 23:10:33.668177 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 30987
I20260805 23:10:33.668325 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971406251336-25971-0/minicluster-data/ts-2/wal/instance
I20260805 23:10:33.668670 25971 external_mini_cluster.cc:1428] Setting key 5a3cf67270595695ec8a109ec01e06f2
I20260805 23:10:33.672180 30696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:33.666957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.195:44463 (local address 127.25.92.254:40757)
0805 23:10:33.667106 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:33.667110 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:33.668352 (+ 1242us) server_negotiation.cc:407] Connection header received
0805 23:10:33.670268 (+ 1916us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:33.670271 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:33.670313 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:33.670410 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:33.671629 (+ 1219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:33.671633 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:33.671636 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:10:33.671656 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:33.671686 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:10:33.671695 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:33.671794 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:33.671970 (+ 176us) server_negotiation.cc:299] Negotiation successful
0805 23:10:33.672021 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260805 23:10:33.673187 31122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:33.667374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:40757 (local address 127.25.92.195:44463)
0805 23:10:33.668148 (+ 774us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:33.668224 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:33.670046 (+ 1822us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:33.670559 (+ 513us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:33.670566 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:33.670667 (+ 101us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:33.671350 (+ 683us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0805 23:10:33.671352 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:33.671395 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0805 23:10:33.671469 (+ 74us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:10:33.671796 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:10:33.671809 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:10:33.671818 (+ 9us) client_negotiation.cc:769] Sending connection context
0805 23:10:33.672007 (+ 189us) client_negotiation.cc:240] Negotiation successful
0805 23:10:33.672282 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":117,"threads_started":1}
I20260805 23:10:33.674394 31120 heartbeater.cc:347] Connected to a master server at 127.25.92.254:40757
I20260805 23:10:33.674656 31120 heartbeater.cc:464] Registering TS with master...
I20260805 23:10:33.675100 31120 heartbeater.cc:511] Master 127.25.92.254:40757 requested a full tablet report, sending...
I20260805 23:10:33.676186 30657 ts_manager.cc:194] Registered new tserver with Master: 664855d22a0a4ce3b49aef4bc3332225 (127.25.92.195:34903)
I20260805 23:10:33.683147 25971 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260805 23:10:33.686986 30696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:33.685161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54378 (local address 127.25.92.254:40757)
0805 23:10:33.685311 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:33.685316 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:33.685518 (+ 202us) server_negotiation.cc:407] Connection header received
0805 23:10:33.685689 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:33.685694 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:33.685735 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:33.685824 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:33.686574 (+ 750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:33.686578 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:33.686581 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:10:33.686593 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:33.686616 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:10:33.686631 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:33.686718 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:33.686792 (+ 74us) server_negotiation.cc:299] Negotiation successful
0805 23:10:33.686836 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260805 23:10:33.689885 30657 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54378:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260805 23:10:33.692911 30657 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260805 23:10:33.705576 31134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:33.702026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.194:39615 (local address 127.0.0.1:49600)
0805 23:10:33.703223 (+ 1197us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:33.703240 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:33.703345 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:33.704385 (+ 1040us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:33.704391 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:33.704408 (+ 17us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:33.704796 (+ 388us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0805 23:10:33.704798 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:33.704810 (+ 12us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0805 23:10:33.704838 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:10:33.705289 (+ 451us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:10:33.705295 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:10:33.705301 (+ 6us) client_negotiation.cc:769] Sending connection context
0805 23:10:33.705373 (+ 72us) client_negotiation.cc:240] Negotiation successful
0805 23:10:33.705404 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1121,"thread_start_us":115,"threads_started":1}
I20260805 23:10:33.705749 31135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:33.702761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32778 (local address 127.25.92.195:34903)
0805 23:10:33.703394 (+ 633us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:33.703408 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:33.703778 (+ 370us) server_negotiation.cc:407] Connection header received
0805 23:10:33.703841 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:33.703848 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:33.704031 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:33.704179 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:33.704853 (+ 674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:33.704860 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:33.704877 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:10:33.704886 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:33.704947 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:10:33.704978 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:33.705100 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:33.705458 (+ 358us) server_negotiation.cc:299] Negotiation successful
0805 23:10:33.705578 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":526,"thread_start_us":108,"threads_started":1}
I20260805 23:10:33.706013 31136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:33.702749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49600 (local address 127.25.92.194:39615)
0805 23:10:33.703181 (+ 432us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:33.703198 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:33.703335 (+ 137us) server_negotiation.cc:407] Connection header received
0805 23:10:33.703450 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:33.703465 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:33.703622 (+ 157us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:33.703750 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:33.704979 (+ 1229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:33.704986 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:33.704998 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:10:33.705015 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:33.705075 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:10:33.705110 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:33.705245 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:33.705787 (+ 542us) server_negotiation.cc:299] Negotiation successful
0805 23:10:33.705852 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":134,"threads_started":1}
I20260805 23:10:33.705956 31131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:33.701877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.193:37877 (local address 127.0.0.1:42036)
0805 23:10:33.702521 (+ 644us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:33.702592 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:33.702704 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:33.703596 (+ 892us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:33.703604 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:33.703689 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:33.704444 (+ 755us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0805 23:10:33.704446 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:33.704470 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0805 23:10:33.704554 (+ 84us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:10:33.705224 (+ 670us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:10:33.705228 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:10:33.705232 (+ 4us) client_negotiation.cc:769] Sending connection context
0805 23:10:33.705318 (+ 86us) client_negotiation.cc:240] Negotiation successful
0805 23:10:33.705365 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":93,"threads_started":1}
I20260805 23:10:33.706544 31132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:33.701998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42036 (local address 127.25.92.193:37877)
0805 23:10:33.702927 (+ 929us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:33.702940 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:33.702966 (+ 26us) server_negotiation.cc:407] Connection header received
0805 23:10:33.703199 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:33.703208 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:33.703413 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:33.703564 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:33.704997 (+ 1433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:33.705002 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:33.705017 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:10:33.705025 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:33.705082 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:10:33.705109 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:33.705221 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:33.705587 (+ 366us) server_negotiation.cc:299] Negotiation successful
0805 23:10:33.705671 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":819,"thread_start_us":121,"threads_started":1}
I20260805 23:10:33.706717 31133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:33.702355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.195:34903 (local address 127.0.0.1:32778)
0805 23:10:33.703584 (+ 1229us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:33.703599 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:33.703676 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:33.704168 (+ 492us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:33.704171 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:33.704188 (+ 17us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:33.704708 (+ 520us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0805 23:10:33.704711 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:33.704721 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0805 23:10:33.704745 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:10:33.705077 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:10:33.705087 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:10:33.705093 (+ 6us) client_negotiation.cc:769] Sending connection context
0805 23:10:33.705208 (+ 115us) client_negotiation.cc:240] Negotiation successful
0805 23:10:33.705300 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1154,"thread_start_us":104,"threads_started":1}
I20260805 23:10:33.709158 30781 tablet_service.cc:1467] Processing CreateTablet for tablet 8df46aa42e9d4714b1c709220bbe5434 (DEFAULT_TABLE table=test-table [id=19405af3d65243ccab8c318b5769281b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:10:33.709271 30915 tablet_service.cc:1467] Processing CreateTablet for tablet 8df46aa42e9d4714b1c709220bbe5434 (DEFAULT_TABLE table=test-table [id=19405af3d65243ccab8c318b5769281b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:10:33.709592 31053 tablet_service.cc:1467] Processing CreateTablet for tablet 8df46aa42e9d4714b1c709220bbe5434 (DEFAULT_TABLE table=test-table [id=19405af3d65243ccab8c318b5769281b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:10:33.710256 30781 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8df46aa42e9d4714b1c709220bbe5434. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:10:33.710347 30915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8df46aa42e9d4714b1c709220bbe5434. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:10:33.710450 31053 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8df46aa42e9d4714b1c709220bbe5434. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:10:33.716917 31137 tablet_bootstrap.cc:492] T 8df46aa42e9d4714b1c709220bbe5434 P 80847a8a70584d3caab29edd59bed44a: Bootstrap starting.
I20260805 23:10:33.718487 31139 tablet_bootstrap.cc:492] T 8df46aa42e9d4714b1c709220bbe5434 P 85f1a8225a5f49a4a222cd567c3d6d66: Bootstrap starting.
I20260805 23:10:33.719739 31137 tablet_bootstrap.cc:654] T 8df46aa42e9d4714b1c709220bbe5434 P 80847a8a70584d3caab29edd59bed44a: Neither blocks nor log segments found. Creating new log.
I20260805 23:10:33.720211 31139 tablet_bootstrap.cc:654] T 8df46aa42e9d4714b1c709220bbe5434 P 85f1a8225a5f49a4a222cd567c3d6d66: Neither blocks nor log segments found. Creating new log.
I20260805 23:10:33.720613 31137 log.cc:824] T 8df46aa42e9d4714b1c709220bbe5434 P 80847a8a70584d3caab29edd59bed44a: Log is configured to *not* fsync() on all Append() calls
I20260805 23:10:33.720933 31139 log.cc:824] T 8df46aa42e9d4714b1c709220bbe5434 P 85f1a8225a5f49a4a222cd567c3d6d66: Log is configured to *not* fsync() on all Append() calls
I20260805 23:10:33.721444 31138 tablet_bootstrap.cc:492] T 8df46aa42e9d4714b1c709220bbe5434 P 664855d22a0a4ce3b49aef4bc3332225: Bootstrap starting.
I20260805 23:10:33.722978 31137 tablet_bootstrap.cc:492] T 8df46aa42e9d4714b1c709220bbe5434 P 80847a8a70584d3caab29edd59bed44a: No bootstrap required, opened a new log
I20260805 23:10:33.722980 31139 tablet_bootstrap.cc:492] T 8df46aa42e9d4714b1c709220bbe5434 P 85f1a8225a5f49a4a222cd567c3d6d66: No bootstrap required, opened a new log
I20260805 23:10:33.723178 31139 ts_tablet_manager.cc:1397] T 8df46aa42e9d4714b1c709220bbe5434 P 85f1a8225a5f49a4a222cd567c3d6d66: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260805 23:10:33.723205 31137 ts_tablet_manager.cc:1397] T 8df46aa42e9d4714b1c709220bbe5434 P 80847a8a70584d3caab29edd59bed44a: Time spent bootstrapping tablet: real 0.006s user 0.006s sys 0.000s
I20260805 23:10:33.723572 31138 tablet_bootstrap.cc:654] T 8df46aa42e9d4714b1c709220bbe5434 P 664855d22a0a4ce3b49aef4bc3332225: Neither blocks nor log segments found. Creating new log.
I20260805 23:10:33.724428 31138 log.cc:824] T 8df46aa42e9d4714b1c709220bbe5434 P 664855d22a0a4ce3b49aef4bc3332225: Log is configured to *not* fsync() on all Append() calls
I20260805 23:10:33.726320 31138 tablet_bootstrap.cc:492] T 8df46aa42e9d4714b1c709220bbe5434 P 664855d22a0a4ce3b49aef4bc3332225: No bootstrap required, opened a new log
I20260805 23:10:33.726540 31138 ts_tablet_manager.cc:1397] T 8df46aa42e9d4714b1c709220bbe5434 P 664855d22a0a4ce3b49aef4bc3332225: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260805 23:10:33.728005 31139 raft_consensus.cc:348] T 8df46aa42e9d4714b1c709220bbe5434 P 85f1a8225a5f49a4a222cd567c3d6d66 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80847a8a70584d3caab29edd59bed44a" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 37877 } } peers { permanent_uuid: "85f1a8225a5f49a4a222cd567c3d6d66" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 39615 } } peers { permanent_uuid: "664855d22a0a4ce3b49aef4bc3332225" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 34903 } }
I20260805 23:10:33.728005 31137 raft_consensus.cc:348] T 8df46aa42e9d4714b1c709220bbe5434 P 80847a8a70584d3caab29edd59bed44a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80847a8a70584d3caab29edd59bed44a" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 37877 } } peers { permanent_uuid: "85f1a8225a5f49a4a222cd567c3d6d66" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 39615 } } peers { permanent_uuid: "664855d22a0a4ce3b49aef4bc3332225" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 34903 } }
I20260805 23:10:33.728297 31139 raft_consensus.cc:374] T 8df46aa42e9d4714b1c709220bbe5434 P 85f1a8225a5f49a4a222cd567c3d6d66 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:10:33.728297 31137 raft_consensus.cc:374] T 8df46aa42e9d4714b1c709220bbe5434 P 80847a8a70584d3caab29edd59bed44a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:10:33.728359 31137 raft_consensus.cc:729] T 8df46aa42e9d4714b1c709220bbe5434 P 80847a8a70584d3caab29edd59bed44a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80847a8a70584d3caab29edd59bed44a, State: Initialized, Role: FOLLOWER
I20260805 23:10:33.728389 31139 raft_consensus.cc:729] T 8df46aa42e9d4714b1c709220bbe5434 P 85f1a8225a5f49a4a222cd567c3d6d66 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85f1a8225a5f49a4a222cd567c3d6d66, State: Initialized, Role: FOLLOWER
I20260805 23:10:33.728897 31137 consensus_queue.cc:260] T 8df46aa42e9d4714b1c709220bbe5434 P 80847a8a70584d3caab29edd59bed44a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80847a8a70584d3caab29edd59bed44a" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 37877 } } peers { permanent_uuid: "85f1a8225a5f49a4a222cd567c3d6d66" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 39615 } } peers { permanent_uuid: "664855d22a0a4ce3b49aef4bc3332225" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 34903 } }
I20260805 23:10:33.728897 31139 consensus_queue.cc:260] T 8df46aa42e9d4714b1c709220bbe5434 P 85f1a8225a5f49a4a222cd567c3d6d66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80847a8a70584d3caab29edd59bed44a" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 37877 } } peers { permanent_uuid: "85f1a8225a5f49a4a222cd567c3d6d66" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 39615 } } peers { permanent_uuid: "664855d22a0a4ce3b49aef4bc3332225" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 34903 } }
I20260805 23:10:33.729996 31137 ts_tablet_manager.cc:1428] T 8df46aa42e9d4714b1c709220bbe5434 P 80847a8a70584d3caab29edd59bed44a: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260805 23:10:33.729996 31139 ts_tablet_manager.cc:1428] T 8df46aa42e9d4714b1c709220bbe5434 P 85f1a8225a5f49a4a222cd567c3d6d66: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260805 23:10:33.730362 30984 heartbeater.cc:503] Master 127.25.92.254:40757 was elected leader, sending a full tablet report...
I20260805 23:10:33.730836 31138 raft_consensus.cc:348] T 8df46aa42e9d4714b1c709220bbe5434 P 664855d22a0a4ce3b49aef4bc3332225 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80847a8a70584d3caab29edd59bed44a" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 37877 } } peers { permanent_uuid: "85f1a8225a5f49a4a222cd567c3d6d66" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 39615 } } peers { permanent_uuid: "664855d22a0a4ce3b49aef4bc3332225" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 34903 } }
I20260805 23:10:33.731052 31138 raft_consensus.cc:374] T 8df46aa42e9d4714b1c709220bbe5434 P 664855d22a0a4ce3b49aef4bc3332225 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:10:33.731112 31138 raft_consensus.cc:729] T 8df46aa42e9d4714b1c709220bbe5434 P 664855d22a0a4ce3b49aef4bc3332225 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 664855d22a0a4ce3b49aef4bc3332225, State: Initialized, Role: FOLLOWER
I20260805 23:10:33.731408 30848 heartbeater.cc:503] Master 127.25.92.254:40757 was elected leader, sending a full tablet report...
I20260805 23:10:33.731648 31138 consensus_queue.cc:260] T 8df46aa42e9d4714b1c709220bbe5434 P 664855d22a0a4ce3b49aef4bc3332225 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80847a8a70584d3caab29edd59bed44a" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 37877 } } peers { permanent_uuid: "85f1a8225a5f49a4a222cd567c3d6d66" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 39615 } } peers { permanent_uuid: "664855d22a0a4ce3b49aef4bc3332225" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 34903 } }
I20260805 23:10:33.732309 31120 heartbeater.cc:503] Master 127.25.92.254:40757 was elected leader, sending a full tablet report...
I20260805 23:10:33.732692 31138 ts_tablet_manager.cc:1428] T 8df46aa42e9d4714b1c709220bbe5434 P 664855d22a0a4ce3b49aef4bc3332225: Time spent starting tablet: real 0.006s user 0.003s sys 0.003s
W20260805 23:10:33.733538 30985 tablet.cc:2369] T 8df46aa42e9d4714b1c709220bbe5434 P 85f1a8225a5f49a4a222cd567c3d6d66: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260805 23:10:33.805665 30849 tablet.cc:2369] T 8df46aa42e9d4714b1c709220bbe5434 P 80847a8a70584d3caab29edd59bed44a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260805 23:10:33.822362 31143 raft_consensus.cc:482] T 8df46aa42e9d4714b1c709220bbe5434 P 85f1a8225a5f49a4a222cd567c3d6d66 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 23:10:33.822558 31143 raft_consensus.cc:504] T 8df46aa42e9d4714b1c709220bbe5434 P 85f1a8225a5f49a4a222cd567c3d6d66 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80847a8a70584d3caab29edd59bed44a" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 37877 } } peers { permanent_uuid: "85f1a8225a5f49a4a222cd567c3d6d66" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 39615 } } peers { permanent_uuid: "664855d22a0a4ce3b49aef4bc3332225" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 34903 } }
I20260805 23:10:33.823709 31143 leader_election.cc:288] T 8df46aa42e9d4714b1c709220bbe5434 P 85f1a8225a5f49a4a222cd567c3d6d66 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80847a8a70584d3caab29edd59bed44a (127.25.92.193:37877), 664855d22a0a4ce3b49aef4bc3332225 (127.25.92.195:34903)
I20260805 23:10:33.825937 31135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:33.824253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.194:54897 (local address 127.25.92.195:34903)
0805 23:10:33.824385 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:33.824388 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:33.824555 (+ 167us) server_negotiation.cc:407] Connection header received
0805 23:10:33.824642 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:33.824644 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:33.824673 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:33.824783 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:33.825400 (+ 617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:33.825405 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:33.825409 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:10:33.825423 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:33.825445 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:10:33.825455 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:33.825595 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:33.825733 (+ 138us) server_negotiation.cc:299] Negotiation successful
0805 23:10:33.825790 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260805 23:10:33.825947 31146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:33.824037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.195:34903 (local address 127.25.92.194:54897)
0805 23:10:33.824464 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:33.824478 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:33.824563 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:33.824783 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:33.824786 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:33.824802 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:33.825207 (+ 405us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0805 23:10:33.825211 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:33.825225 (+ 14us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0805 23:10:33.825259 (+ 34us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:10:33.825631 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:10:33.825635 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:10:33.825639 (+ 4us) client_negotiation.cc:769] Sending connection context
0805 23:10:33.825731 (+ 92us) client_negotiation.cc:240] Negotiation successful
0805 23:10:33.825774 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":120,"threads_started":1}
I20260805 23:10:33.826053 31132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:33.824054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.194:45965 (local address 127.25.92.193:37877)
0805 23:10:33.824232 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:33.824237 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:33.824255 (+ 18us) server_negotiation.cc:407] Connection header received
0805 23:10:33.824306 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:33.824310 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:33.824351 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:33.824441 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:33.825649 (+ 1208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:33.825653 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:33.825655 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:10:33.825672 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:33.825697 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:10:33.825710 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:33.825791 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:33.825872 (+ 81us) server_negotiation.cc:299] Negotiation successful
0805 23:10:33.825910 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260805 23:10:33.826270 30986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:33.823919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.193:37877 (local address 127.25.92.194:45965)
0805 23:10:33.824094 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:33.824111 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:33.824204 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:33.824550 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:33.824553 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:33.824565 (+ 12us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:33.825437 (+ 872us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0805 23:10:33.825439 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:33.825449 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0805 23:10:33.825475 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:10:33.825792 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:10:33.825795 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:10:33.825798 (+ 3us) client_negotiation.cc:769] Sending connection context
0805 23:10:33.825877 (+ 79us) client_negotiation.cc:240] Negotiation successful
0805 23:10:33.825908 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260805 23:10:33.826686 31073 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8df46aa42e9d4714b1c709220bbe5434" candidate_uuid: "85f1a8225a5f49a4a222cd567c3d6d66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "664855d22a0a4ce3b49aef4bc3332225" is_pre_election: true
I20260805 23:10:33.826879 30802 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8df46aa42e9d4714b1c709220bbe5434" candidate_uuid: "85f1a8225a5f49a4a222cd567c3d6d66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80847a8a70584d3caab29edd59bed44a" is_pre_election: true
I20260805 23:10:33.827042 31073 raft_consensus.cc:2454] T 8df46aa42e9d4714b1c709220bbe5434 P 664855d22a0a4ce3b49aef4bc3332225 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85f1a8225a5f49a4a222cd567c3d6d66 in term 0.
I20260805 23:10:33.827136 30802 raft_consensus.cc:2454] T 8df46aa42e9d4714b1c709220bbe5434 P 80847a8a70584d3caab29edd59bed44a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85f1a8225a5f49a4a222cd567c3d6d66 in term 0.
I20260805 23:10:33.827555 30870 leader_election.cc:302] T 8df46aa42e9d4714b1c709220bbe5434 P 85f1a8225a5f49a4a222cd567c3d6d66 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 664855d22a0a4ce3b49aef4bc3332225, 80847a8a70584d3caab29edd59bed44a, 85f1a8225a5f49a4a222cd567c3d6d66; no voters:
I20260805 23:10:33.827847 31143 raft_consensus.cc:2781] T 8df46aa42e9d4714b1c709220bbe5434 P 85f1a8225a5f49a4a222cd567c3d6d66 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260805 23:10:33.827956 31143 raft_consensus.cc:482] T 8df46aa42e9d4714b1c709220bbe5434 P 85f1a8225a5f49a4a222cd567c3d6d66 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260805 23:10:33.828014 31143 raft_consensus.cc:3037] T 8df46aa42e9d4714b1c709220bbe5434 P 85f1a8225a5f49a4a222cd567c3d6d66 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:10:33.829214 31143 raft_consensus.cc:504] T 8df46aa42e9d4714b1c709220bbe5434 P 85f1a8225a5f49a4a222cd567c3d6d66 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80847a8a70584d3caab29edd59bed44a" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 37877 } } peers { permanent_uuid: "85f1a8225a5f49a4a222cd567c3d6d66" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 39615 } } peers { permanent_uuid: "664855d22a0a4ce3b49aef4bc3332225" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 34903 } }
I20260805 23:10:33.829617 31143 leader_election.cc:288] T 8df46aa42e9d4714b1c709220bbe5434 P 85f1a8225a5f49a4a222cd567c3d6d66 [CANDIDATE]: Term 1 election: Requested vote from peers 80847a8a70584d3caab29edd59bed44a (127.25.92.193:37877), 664855d22a0a4ce3b49aef4bc3332225 (127.25.92.195:34903)
I20260805 23:10:33.829962 31073 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8df46aa42e9d4714b1c709220bbe5434" candidate_uuid: "85f1a8225a5f49a4a222cd567c3d6d66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "664855d22a0a4ce3b49aef4bc3332225"
I20260805 23:10:33.829985 30802 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8df46aa42e9d4714b1c709220bbe5434" candidate_uuid: "85f1a8225a5f49a4a222cd567c3d6d66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80847a8a70584d3caab29edd59bed44a"
I20260805 23:10:33.830075 31073 raft_consensus.cc:3037] T 8df46aa42e9d4714b1c709220bbe5434 P 664855d22a0a4ce3b49aef4bc3332225 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:10:33.830125 30802 raft_consensus.cc:3037] T 8df46aa42e9d4714b1c709220bbe5434 P 80847a8a70584d3caab29edd59bed44a [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:10:33.831130 31073 raft_consensus.cc:2454] T 8df46aa42e9d4714b1c709220bbe5434 P 664855d22a0a4ce3b49aef4bc3332225 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85f1a8225a5f49a4a222cd567c3d6d66 in term 1.
I20260805 23:10:33.831213 30802 raft_consensus.cc:2454] T 8df46aa42e9d4714b1c709220bbe5434 P 80847a8a70584d3caab29edd59bed44a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85f1a8225a5f49a4a222cd567c3d6d66 in term 1.
I20260805 23:10:33.831461 30868 leader_election.cc:302] T 8df46aa42e9d4714b1c709220bbe5434 P 85f1a8225a5f49a4a222cd567c3d6d66 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 664855d22a0a4ce3b49aef4bc3332225, 85f1a8225a5f49a4a222cd567c3d6d66; no voters:
I20260805 23:10:33.831640 31143 raft_consensus.cc:2781] T 8df46aa42e9d4714b1c709220bbe5434 P 85f1a8225a5f49a4a222cd567c3d6d66 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:10:33.831916 31143 raft_consensus.cc:686] T 8df46aa42e9d4714b1c709220bbe5434 P 85f1a8225a5f49a4a222cd567c3d6d66 [term 1 LEADER]: Becoming Leader. State: Replica: 85f1a8225a5f49a4a222cd567c3d6d66, State: Running, Role: LEADER
I20260805 23:10:33.832475 31143 consensus_queue.cc:237] T 8df46aa42e9d4714b1c709220bbe5434 P 85f1a8225a5f49a4a222cd567c3d6d66 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80847a8a70584d3caab29edd59bed44a" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 37877 } } peers { permanent_uuid: "85f1a8225a5f49a4a222cd567c3d6d66" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 39615 } } peers { permanent_uuid: "664855d22a0a4ce3b49aef4bc3332225" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 34903 } }
I20260805 23:10:33.835995 30656 catalog_manager.cc:5495] T 8df46aa42e9d4714b1c709220bbe5434 P 85f1a8225a5f49a4a222cd567c3d6d66 reported cstate change: term changed from 0 to 1, leader changed from <none> to 85f1a8225a5f49a4a222cd567c3d6d66 (127.25.92.194). New cstate: current_term: 1 leader_uuid: "85f1a8225a5f49a4a222cd567c3d6d66" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80847a8a70584d3caab29edd59bed44a" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 37877 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85f1a8225a5f49a4a222cd567c3d6d66" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 39615 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "664855d22a0a4ce3b49aef4bc3332225" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 34903 } health_report { overall_health: UNKNOWN } } }
I20260805 23:10:33.865408 31136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:33.863647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49602 (local address 127.25.92.194:39615)
0805 23:10:33.863798 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:33.863802 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:33.863817 (+ 15us) server_negotiation.cc:407] Connection header received
0805 23:10:33.864006 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:33.864008 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:33.864071 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:33.864161 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:33.864959 (+ 798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:33.864962 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:33.864964 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:10:33.864981 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:33.864999 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:10:33.865008 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:33.865101 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:33.865203 (+ 102us) server_negotiation.cc:299] Negotiation successful
0805 23:10:33.865245 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260805 23:10:33.871541 30802 raft_consensus.cc:1261] T 8df46aa42e9d4714b1c709220bbe5434 P 80847a8a70584d3caab29edd59bed44a [term 1 FOLLOWER]: Refusing update from remote peer 85f1a8225a5f49a4a222cd567c3d6d66: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:10:33.871538 31073 raft_consensus.cc:1261] T 8df46aa42e9d4714b1c709220bbe5434 P 664855d22a0a4ce3b49aef4bc3332225 [term 1 FOLLOWER]: Refusing update from remote peer 85f1a8225a5f49a4a222cd567c3d6d66: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:10:33.872269 31147 consensus_queue.cc:1035] T 8df46aa42e9d4714b1c709220bbe5434 P 85f1a8225a5f49a4a222cd567c3d6d66 [LEADER]: Connected to new peer: Peer: permanent_uuid: "664855d22a0a4ce3b49aef4bc3332225" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 34903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:10:33.872553 31143 consensus_queue.cc:1035] T 8df46aa42e9d4714b1c709220bbe5434 P 85f1a8225a5f49a4a222cd567c3d6d66 [LEADER]: Connected to new peer: Peer: permanent_uuid: "80847a8a70584d3caab29edd59bed44a" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 37877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:10:33.890034 30657 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:54378:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:10:33.890254 30657 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:54378:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:10:33.892824 30657 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6af8d513de3142a5be8057f17a7dd6b4: Sending DeleteTablet for 3 replicas of tablet 8df46aa42e9d4714b1c709220bbe5434
I20260805 23:10:33.893721 30915 tablet_service.cc:1514] Processing DeleteTablet for tablet 8df46aa42e9d4714b1c709220bbe5434 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:10:33 UTC) from {username='slave'} at 127.0.0.1:49600
I20260805 23:10:33.893922 31053 tablet_service.cc:1514] Processing DeleteTablet for tablet 8df46aa42e9d4714b1c709220bbe5434 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:10:33 UTC) from {username='slave'} at 127.0.0.1:32778
I20260805 23:10:33.894302 30781 tablet_service.cc:1514] Processing DeleteTablet for tablet 8df46aa42e9d4714b1c709220bbe5434 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:10:33 UTC) from {username='slave'} at 127.0.0.1:42036
I20260805 23:10:33.894596 31161 tablet_replica.cc:330] T 8df46aa42e9d4714b1c709220bbe5434 P 85f1a8225a5f49a4a222cd567c3d6d66: stopping tablet replica
I20260805 23:10:33.894665 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 30719
I20260805 23:10:33.894955 31161 raft_consensus.cc:2229] T 8df46aa42e9d4714b1c709220bbe5434 P 85f1a8225a5f49a4a222cd567c3d6d66 [term 1 LEADER]: Raft consensus shutting down.
I20260805 23:10:33.895376 31161 raft_consensus.cc:2258] T 8df46aa42e9d4714b1c709220bbe5434 P 85f1a8225a5f49a4a222cd567c3d6d66 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:10:33.896291 31162 tablet_replica.cc:330] T 8df46aa42e9d4714b1c709220bbe5434 P 664855d22a0a4ce3b49aef4bc3332225: stopping tablet replica
I20260805 23:10:33.896692 31162 raft_consensus.cc:2229] T 8df46aa42e9d4714b1c709220bbe5434 P 664855d22a0a4ce3b49aef4bc3332225 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 23:10:33.897073 31161 ts_tablet_manager.cc:1905] T 8df46aa42e9d4714b1c709220bbe5434 P 85f1a8225a5f49a4a222cd567c3d6d66: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:10:33.897091 31162 raft_consensus.cc:2258] T 8df46aa42e9d4714b1c709220bbe5434 P 664855d22a0a4ce3b49aef4bc3332225 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:10:33.898603 31162 ts_tablet_manager.cc:1905] T 8df46aa42e9d4714b1c709220bbe5434 P 664855d22a0a4ce3b49aef4bc3332225: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:10:33.900463 31161 ts_tablet_manager.cc:1918] T 8df46aa42e9d4714b1c709220bbe5434 P 85f1a8225a5f49a4a222cd567c3d6d66: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 23:10:33.900592 31161 log.cc:1196] T 8df46aa42e9d4714b1c709220bbe5434 P 85f1a8225a5f49a4a222cd567c3d6d66: Deleting WAL directory at /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971406251336-25971-0/minicluster-data/ts-1/wal/wals/8df46aa42e9d4714b1c709220bbe5434
I20260805 23:10:33.901022 31161 ts_tablet_manager.cc:1939] T 8df46aa42e9d4714b1c709220bbe5434 P 85f1a8225a5f49a4a222cd567c3d6d66: Deleting consensus metadata
I20260805 23:10:33.901746 31162 ts_tablet_manager.cc:1918] T 8df46aa42e9d4714b1c709220bbe5434 P 664855d22a0a4ce3b49aef4bc3332225: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 23:10:33.901856 31162 log.cc:1196] T 8df46aa42e9d4714b1c709220bbe5434 P 664855d22a0a4ce3b49aef4bc3332225: Deleting WAL directory at /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971406251336-25971-0/minicluster-data/ts-2/wal/wals/8df46aa42e9d4714b1c709220bbe5434
I20260805 23:10:33.902271 31162 ts_tablet_manager.cc:1939] T 8df46aa42e9d4714b1c709220bbe5434 P 664855d22a0a4ce3b49aef4bc3332225: Deleting consensus metadata
I20260805 23:10:33.902496 30641 catalog_manager.cc:4841] TS 85f1a8225a5f49a4a222cd567c3d6d66 (127.25.92.194:39615): tablet 8df46aa42e9d4714b1c709220bbe5434 (table test-table [id=19405af3d65243ccab8c318b5769281b]) successfully deleted
I20260805 23:10:33.903381 30642 catalog_manager.cc:4841] TS 664855d22a0a4ce3b49aef4bc3332225 (127.25.92.195:34903): tablet 8df46aa42e9d4714b1c709220bbe5434 (table test-table [id=19405af3d65243ccab8c318b5769281b]) successfully deleted
I20260805 23:10:33.913722 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 30851
W20260805 23:10:33.913434 30644 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.25.92.193:37877 (error 108)
I20260805 23:10:33.919080 31133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:33.918618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.193:37877 (local address 127.0.0.1:42048)
0805 23:10:33.918807 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:33.918943 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.193:37877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260805 23:10:33.919458 30644 catalog_manager.cc:4584] TS 80847a8a70584d3caab29edd59bed44a (127.25.92.193:37877): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8df46aa42e9d4714b1c709220bbe5434: Network error: Client connection negotiation failed: client connection to 127.25.92.193:37877: connect: Connection refused (error 111)
I20260805 23:10:33.923667 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 30987
I20260805 23:10:33.931221 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 30626
2026-08-05T23:10:33Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (1072 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260805 23:10:33.953781 25971 mini_oidc.cc:152] Starting JWKS server
I20260805 23:10:33.955514 25971 webserver.cc:460] Webserver started at https://127.0.0.1:35423/ using document root <none> and password file <none>
I20260805 23:10:33.955705 25971 mini_oidc.cc:189] Starting OIDC Discovery server
I20260805 23:10:33.955984 25971 webserver.cc:460] Webserver started at http://127.0.0.1:33275/ using document root <none> and password file <none>
2026-08-05T23:10:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:10:33Z Disabled control of system clock
I20260805 23:10:33.965154 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785971406251336-25971-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785971406251336-25971-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785971406251336-25971-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.254:44591
--webserver_interface=127.25.92.254
--webserver_port=0
--builtin_ntp_servers=127.25.92.212:41341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.92.254:44591
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35423/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785971406251336-25971-0/testchainca.cert with env {}
W20260805 23:10:34.076046 31170 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:10:34.076424 31170 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:10:34.076504 31170 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:10:34.080777 31170 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:10:34.080896 31170 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:10:34.080928 31170 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:10:34.080972 31170 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:10:34.081005 31170 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260805 23:10:34.081050 31170 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260805 23:10:34.081084 31170 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:35423/jwks.json
I20260805 23:10:34.086910 31170 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:41341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785971406251336-25971-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785971406251336-25971-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.92.254:44591
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.254:44591
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35423/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.92.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785971406251336-25971-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31170
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785971406251336-25971-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:10:34.088146 31170 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:10:34.089246 31170 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:10:34.098125 31175 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:34.098157 31176 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:34.098125 31178 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:10:34.098217 31170 server_base.cc:1034] running on GCE node
I20260805 23:10:34.099071 31170 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:10:34.100230 31170 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:10:34.101423 31170 hybrid_clock.cc:648] HybridClock initialized: now 1785971434101406 us; error 37 us; skew 500 ppm
I20260805 23:10:34.103647 31170 webserver.cc:460] Webserver started at http://127.25.92.254:33991/ using document root <none> and password file <none>
I20260805 23:10:34.104274 31170 fs_manager.cc:359] Metadata directory not provided
I20260805 23:10:34.104380 31170 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:10:34.104616 31170 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:10:34.106745 31170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785971406251336-25971-0/minicluster-data/master-0/data/instance:
uuid: "7d7c4071ca064f23ac8bbd05ca08d77c"
format_stamp: "Formatted at 2026-08-05 23:10:34 on dist-test-slave-ss4n"
server_key: "c785d8a2bf4d5f0e7e13eb16ff111e15"
server_key_iv: "cb48426d441813b82a885325d24dcd9b"
server_key_version: "encryptionkey@0"
I20260805 23:10:34.107329 31170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785971406251336-25971-0/minicluster-data/master-0/wal/instance:
uuid: "7d7c4071ca064f23ac8bbd05ca08d77c"
format_stamp: "Formatted at 2026-08-05 23:10:34 on dist-test-slave-ss4n"
server_key: "c785d8a2bf4d5f0e7e13eb16ff111e15"
server_key_iv: "cb48426d441813b82a885325d24dcd9b"
server_key_version: "encryptionkey@0"
I20260805 23:10:34.111330 31170 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260805 23:10:34.114033 31184 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:34.115392 31170 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260805 23:10:34.115496 31170 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785971406251336-25971-0/minicluster-data/master-0/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785971406251336-25971-0/minicluster-data/master-0/wal
uuid: "7d7c4071ca064f23ac8bbd05ca08d77c"
format_stamp: "Formatted at 2026-08-05 23:10:34 on dist-test-slave-ss4n"
server_key: "c785d8a2bf4d5f0e7e13eb16ff111e15"
server_key_iv: "cb48426d441813b82a885325d24dcd9b"
server_key_version: "encryptionkey@0"
I20260805 23:10:34.115599 31170 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785971406251336-25971-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785971406251336-25971-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785971406251336-25971-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:10:34.125607 31170 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:10:34.126252 31170 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:10:34.126410 31170 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:10:34.133580 31170 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.254:44591
I20260805 23:10:34.133587 31237 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.254:44591 every 8 connection(s)
I20260805 23:10:34.134542 31170 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
I20260805 23:10:34.137801 31238 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:10:34.141302 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 31170
I20260805 23:10:34.141404 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785971406251336-25971-0/minicluster-data/master-0/wal/instance
I20260805 23:10:34.141798 25971 external_mini_cluster.cc:1428] Setting key edaff288956775245439c13cd53b343f
I20260805 23:10:34.143894 31238 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d7c4071ca064f23ac8bbd05ca08d77c: Bootstrap starting.
I20260805 23:10:34.146446 31238 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7d7c4071ca064f23ac8bbd05ca08d77c: Neither blocks nor log segments found. Creating new log.
I20260805 23:10:34.147217 31238 log.cc:824] T 00000000000000000000000000000000 P 7d7c4071ca064f23ac8bbd05ca08d77c: Log is configured to *not* fsync() on all Append() calls
I20260805 23:10:34.149207 31238 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d7c4071ca064f23ac8bbd05ca08d77c: No bootstrap required, opened a new log
I20260805 23:10:34.152319 31241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:34.143160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33862 (local address 127.25.92.254:44591)
0805 23:10:34.143913 (+ 753us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:34.143935 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:34.143977 (+ 42us) server_negotiation.cc:407] Connection header received
0805 23:10:34.144864 (+ 887us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:34.144884 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:34.145317 (+ 433us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:34.145717 (+ 400us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:34.146100 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:34.147242 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:34.148276 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:34.148812 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:34.149022 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:34.150010 (+ 988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:34.150016 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:34.150033 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:10:34.150570 (+ 537us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:34.150646 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:10:34.150674 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:34.150781 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:34.150969 (+ 188us) server_negotiation.cc:299] Negotiation successful
0805 23:10:34.151304 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":286,"thread_start_us":142,"threads_started":1}
I20260805 23:10:34.153925 31238 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7d7c4071ca064f23ac8bbd05ca08d77c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d7c4071ca064f23ac8bbd05ca08d77c" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 44591 } }
I20260805 23:10:34.154165 31238 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7d7c4071ca064f23ac8bbd05ca08d77c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:10:34.154222 31238 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7d7c4071ca064f23ac8bbd05ca08d77c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d7c4071ca064f23ac8bbd05ca08d77c, State: Initialized, Role: FOLLOWER
I20260805 23:10:34.154711 31238 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7d7c4071ca064f23ac8bbd05ca08d77c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d7c4071ca064f23ac8bbd05ca08d77c" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 44591 } }
I20260805 23:10:34.154846 31238 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7d7c4071ca064f23ac8bbd05ca08d77c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:10:34.154933 31238 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7d7c4071ca064f23ac8bbd05ca08d77c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:10:34.155023 31238 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7d7c4071ca064f23ac8bbd05ca08d77c [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:10:34.155835 25971 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260805 23:10:34.156313 31238 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7d7c4071ca064f23ac8bbd05ca08d77c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d7c4071ca064f23ac8bbd05ca08d77c" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 44591 } }
I20260805 23:10:34.156682 31238 leader_election.cc:302] T 00000000000000000000000000000000 P 7d7c4071ca064f23ac8bbd05ca08d77c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7d7c4071ca064f23ac8bbd05ca08d77c; no voters:
I20260805 23:10:34.157037 31238 leader_election.cc:288] T 00000000000000000000000000000000 P 7d7c4071ca064f23ac8bbd05ca08d77c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 23:10:34.157521 31244 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7d7c4071ca064f23ac8bbd05ca08d77c [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:10:34.157842 31244 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7d7c4071ca064f23ac8bbd05ca08d77c [term 1 LEADER]: Becoming Leader. State: Replica: 7d7c4071ca064f23ac8bbd05ca08d77c, State: Running, Role: LEADER
I20260805 23:10:34.158171 31244 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7d7c4071ca064f23ac8bbd05ca08d77c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d7c4071ca064f23ac8bbd05ca08d77c" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 44591 } }
I20260805 23:10:34.158468 31238 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7d7c4071ca064f23ac8bbd05ca08d77c [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:10:34.159309 31244 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d7c4071ca064f23ac8bbd05ca08d77c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7d7c4071ca064f23ac8bbd05ca08d77c. Latest consensus state: current_term: 1 leader_uuid: "7d7c4071ca064f23ac8bbd05ca08d77c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d7c4071ca064f23ac8bbd05ca08d77c" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 44591 } } }
I20260805 23:10:34.159438 31244 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d7c4071ca064f23ac8bbd05ca08d77c [sys.catalog]: This master's current role is: LEADER
I20260805 23:10:34.159739 31243 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d7c4071ca064f23ac8bbd05ca08d77c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7d7c4071ca064f23ac8bbd05ca08d77c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d7c4071ca064f23ac8bbd05ca08d77c" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 44591 } } }
I20260805 23:10:34.159830 31243 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d7c4071ca064f23ac8bbd05ca08d77c [sys.catalog]: This master's current role is: LEADER
I20260805 23:10:34.159874 31252 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:10:34.163021 31252 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:10:34.170599 31252 catalog_manager.cc:1347] Generated new cluster ID: 05c29993fd5c42cf84db631692e03705
I20260805 23:10:34.170686 31252 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:10:34.193933 31252 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:10:34.194978 31252 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:10:34.201613 31252 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7d7c4071ca064f23ac8bbd05ca08d77c: Generated new TSK 0
I20260805 23:10:34.202384 31252 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260805 23:10:34.238909 31268 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260805 23:10:34.239099 31268 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260805 23:10:34.239741 31241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:34.219602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33866 (local address 127.25.92.254:44591)
0805 23:10:34.219753 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:34.219758 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:34.219986 (+ 228us) server_negotiation.cc:407] Connection header received
0805 23:10:34.220217 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:34.220221 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:34.220282 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:34.220429 (+ 147us) server_negotiation.cc:226] Negotiated authn=JWT
0805 23:10:34.220733 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:34.221624 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:34.222667 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:34.223045 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:34.223213 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:34.223579 (+ 366us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0805 23:10:34.239289 (+ 15710us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35423/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0805 23:10:34.239598 (+ 309us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:33866: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35423/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":54}
W20260805 23:10:34.239835 31241 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:33866: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35423/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260805 23:10:34.241155 31267 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:10:34.219522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:44591 (local address 127.0.0.1:33866)
0805 23:10:34.219907 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:34.219921 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:34.220040 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:34.220478 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:34.220481 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:34.220489 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0805 23:10:34.220610 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:34.220615 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:34.221887 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:34.221891 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:34.222521 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:34.222529 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:34.223208 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:34.223212 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:34.223288 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:34.223335 (+ 47us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0805 23:10:34.239575 (+ 16240us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35423/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0805 23:10:34.239827 (+ 252us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.25.92.254:44591: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35423/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":118,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.25.92.254:44591: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35423/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260805 23:10:34.241999 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 31170
2026-08-05T23:10:34Z chronyd exiting
I20260805 23:10:34.280798 25971 test_util.cc:175] -----------------------------------------------
I20260805 23:10:34.280922 25971 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785971406251336-25971-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (329 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260805 23:10:34.282332 25971 mini_oidc.cc:152] Starting JWKS server
I20260805 23:10:34.283354 25971 webserver.cc:460] Webserver started at https://127.0.0.1:34277/ using document root <none> and password file <none>
I20260805 23:10:34.283519 25971 mini_oidc.cc:189] Starting OIDC Discovery server
I20260805 23:10:34.283834 25971 webserver.cc:460] Webserver started at http://127.0.0.1:32973/ using document root <none> and password file <none>
2026-08-05T23:10:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:10:34Z Disabled control of system clock
I20260805 23:10:34.294341 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785971406251336-25971-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785971406251336-25971-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785971406251336-25971-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.254:44183
--webserver_interface=127.25.92.254
--webserver_port=0
--builtin_ntp_servers=127.25.92.212:38443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.92.254:44183
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34277/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785971406251336-25971-0/testchainca.cert with env {}
W20260805 23:10:34.411602 31274 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:10:34.411914 31274 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:10:34.411960 31274 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:10:34.416674 31274 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:10:34.416760 31274 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:10:34.416778 31274 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:10:34.416798 31274 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:10:34.416815 31274 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260805 23:10:34.416872 31274 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260805 23:10:34.416901 31274 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:34277/jwks.json
I20260805 23:10:34.423353 31274 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:38443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785971406251336-25971-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785971406251336-25971-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.92.254:44183
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.254:44183
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34277/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.92.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785971406251336-25971-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31274
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785971406251336-25971-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:10:34.424625 31274 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:10:34.425783 31274 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:10:34.435567 31282 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:34.435571 31279 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:10:34.435654 31274 server_base.cc:1034] running on GCE node
W20260805 23:10:34.435725 31280 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:10:34.436427 31274 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:10:34.437589 31274 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:10:34.438800 31274 hybrid_clock.cc:648] HybridClock initialized: now 1785971434438787 us; error 27 us; skew 500 ppm
I20260805 23:10:34.440866 31274 webserver.cc:460] Webserver started at http://127.25.92.254:35183/ using document root <none> and password file <none>
I20260805 23:10:34.441545 31274 fs_manager.cc:359] Metadata directory not provided
I20260805 23:10:34.441638 31274 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:10:34.441869 31274 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:10:34.444039 31274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785971406251336-25971-0/minicluster-data/master-0/data/instance:
uuid: "03bbd151e9fc463992d768db6b19c49b"
format_stamp: "Formatted at 2026-08-05 23:10:34 on dist-test-slave-ss4n"
server_key: "c65664a2bcdadb4cd86b124eace422bc"
server_key_iv: "1c71f0a8093fb36a8d7b369126a0f666"
server_key_version: "encryptionkey@0"
I20260805 23:10:34.444628 31274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785971406251336-25971-0/minicluster-data/master-0/wal/instance:
uuid: "03bbd151e9fc463992d768db6b19c49b"
format_stamp: "Formatted at 2026-08-05 23:10:34 on dist-test-slave-ss4n"
server_key: "c65664a2bcdadb4cd86b124eace422bc"
server_key_iv: "1c71f0a8093fb36a8d7b369126a0f666"
server_key_version: "encryptionkey@0"
I20260805 23:10:34.448504 31274 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260805 23:10:34.451228 31288 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:34.452551 31274 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260805 23:10:34.452704 31274 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785971406251336-25971-0/minicluster-data/master-0/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785971406251336-25971-0/minicluster-data/master-0/wal
uuid: "03bbd151e9fc463992d768db6b19c49b"
format_stamp: "Formatted at 2026-08-05 23:10:34 on dist-test-slave-ss4n"
server_key: "c65664a2bcdadb4cd86b124eace422bc"
server_key_iv: "1c71f0a8093fb36a8d7b369126a0f666"
server_key_version: "encryptionkey@0"
I20260805 23:10:34.452831 31274 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785971406251336-25971-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785971406251336-25971-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785971406251336-25971-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:10:34.473131 31274 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:10:34.473922 31274 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:10:34.474140 31274 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:10:34.481696 31274 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.254:44183
I20260805 23:10:34.481720 31341 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.254:44183 every 8 connection(s)
I20260805 23:10:34.482849 31274 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
I20260805 23:10:34.486495 31342 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:10:34.491205 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 31274
I20260805 23:10:34.491360 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785971406251336-25971-0/minicluster-data/master-0/wal/instance
I20260805 23:10:34.491698 25971 external_mini_cluster.cc:1428] Setting key ec7c4e8896f0f166f241386486ce0896
I20260805 23:10:34.492843 31342 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03bbd151e9fc463992d768db6b19c49b: Bootstrap starting.
I20260805 23:10:34.495307 31342 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 03bbd151e9fc463992d768db6b19c49b: Neither blocks nor log segments found. Creating new log.
I20260805 23:10:34.496235 31342 log.cc:824] T 00000000000000000000000000000000 P 03bbd151e9fc463992d768db6b19c49b: Log is configured to *not* fsync() on all Append() calls
I20260805 23:10:34.498739 31342 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03bbd151e9fc463992d768db6b19c49b: No bootstrap required, opened a new log
I20260805 23:10:34.502401 31345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:34.493312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46134 (local address 127.25.92.254:44183)
0805 23:10:34.494035 (+ 723us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:34.494052 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:34.494089 (+ 37us) server_negotiation.cc:407] Connection header received
0805 23:10:34.494799 (+ 710us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:34.494817 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:34.495226 (+ 409us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:34.495589 (+ 363us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:34.495998 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:34.497227 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:34.498497 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:34.499033 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:34.499167 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:34.500237 (+ 1070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:34.500243 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:34.500258 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:10:34.500712 (+ 454us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:34.500842 (+ 130us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:10:34.500876 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:34.500969 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:34.501255 (+ 286us) server_negotiation.cc:299] Negotiation successful
0805 23:10:34.501605 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":210,"threads_started":1}
I20260805 23:10:34.503198 31342 raft_consensus.cc:348] T 00000000000000000000000000000000 P 03bbd151e9fc463992d768db6b19c49b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03bbd151e9fc463992d768db6b19c49b" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 44183 } }
I20260805 23:10:34.503484 31342 raft_consensus.cc:374] T 00000000000000000000000000000000 P 03bbd151e9fc463992d768db6b19c49b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:10:34.503573 31342 raft_consensus.cc:729] T 00000000000000000000000000000000 P 03bbd151e9fc463992d768db6b19c49b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03bbd151e9fc463992d768db6b19c49b, State: Initialized, Role: FOLLOWER
I20260805 23:10:34.504027 31342 consensus_queue.cc:260] T 00000000000000000000000000000000 P 03bbd151e9fc463992d768db6b19c49b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03bbd151e9fc463992d768db6b19c49b" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 44183 } }
I20260805 23:10:34.504179 31342 raft_consensus.cc:388] T 00000000000000000000000000000000 P 03bbd151e9fc463992d768db6b19c49b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:10:34.504252 31342 raft_consensus.cc:482] T 00000000000000000000000000000000 P 03bbd151e9fc463992d768db6b19c49b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:10:34.504348 31342 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 03bbd151e9fc463992d768db6b19c49b [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:10:34.505451 31342 raft_consensus.cc:504] T 00000000000000000000000000000000 P 03bbd151e9fc463992d768db6b19c49b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03bbd151e9fc463992d768db6b19c49b" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 44183 } }
I20260805 23:10:34.505649 25971 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260805 23:10:34.505854 31342 leader_election.cc:302] T 00000000000000000000000000000000 P 03bbd151e9fc463992d768db6b19c49b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 03bbd151e9fc463992d768db6b19c49b; no voters:
I20260805 23:10:34.506208 31342 leader_election.cc:288] T 00000000000000000000000000000000 P 03bbd151e9fc463992d768db6b19c49b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 23:10:34.506285 31347 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 03bbd151e9fc463992d768db6b19c49b [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:10:34.506487 31347 raft_consensus.cc:686] T 00000000000000000000000000000000 P 03bbd151e9fc463992d768db6b19c49b [term 1 LEADER]: Becoming Leader. State: Replica: 03bbd151e9fc463992d768db6b19c49b, State: Running, Role: LEADER
I20260805 23:10:34.506850 31347 consensus_queue.cc:237] T 00000000000000000000000000000000 P 03bbd151e9fc463992d768db6b19c49b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03bbd151e9fc463992d768db6b19c49b" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 44183 } }
I20260805 23:10:34.507566 31342 sys_catalog.cc:564] T 00000000000000000000000000000000 P 03bbd151e9fc463992d768db6b19c49b [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:10:34.508101 31348 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03bbd151e9fc463992d768db6b19c49b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 03bbd151e9fc463992d768db6b19c49b. Latest consensus state: current_term: 1 leader_uuid: "03bbd151e9fc463992d768db6b19c49b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03bbd151e9fc463992d768db6b19c49b" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 44183 } } }
I20260805 23:10:34.508235 31348 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03bbd151e9fc463992d768db6b19c49b [sys.catalog]: This master's current role is: LEADER
I20260805 23:10:34.508637 31352 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:10:34.508882 31349 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03bbd151e9fc463992d768db6b19c49b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "03bbd151e9fc463992d768db6b19c49b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03bbd151e9fc463992d768db6b19c49b" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 44183 } } }
I20260805 23:10:34.508992 31349 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03bbd151e9fc463992d768db6b19c49b [sys.catalog]: This master's current role is: LEADER
I20260805 23:10:34.511933 31352 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:10:34.517997 31352 catalog_manager.cc:1347] Generated new cluster ID: ec914749dec14c6d9e5742498aef887b
I20260805 23:10:34.518095 31352 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:10:34.533984 31352 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:10:34.534976 31352 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:10:34.546443 31352 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 03bbd151e9fc463992d768db6b19c49b: Generated new TSK 0
I20260805 23:10:34.547223 31352 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260805 23:10:34.579926 31372 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260805 23:10:34.580039 31372 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260805 23:10:34.580797 31345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:34.563522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46142 (local address 127.25.92.254:44183)
0805 23:10:34.563670 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:34.563675 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:34.563939 (+ 264us) server_negotiation.cc:407] Connection header received
0805 23:10:34.564147 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:34.564150 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:34.564212 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:34.564346 (+ 134us) server_negotiation.cc:226] Negotiated authn=JWT
0805 23:10:34.564582 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:34.565442 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:34.566460 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:34.566814 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:34.566923 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:34.567275 (+ 352us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0805 23:10:34.580328 (+ 13053us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34277/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0805 23:10:34.580633 (+ 305us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:46142: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34277/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":55}
W20260805 23:10:34.580873 31345 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:46142: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34277/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260805 23:10:34.580927 31371 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:10:34.563390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:44183 (local address 127.0.0.1:46142)
0805 23:10:34.563816 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:34.563837 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:34.563988 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:34.564352 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:34.564354 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:34.564361 (+ 7us) client_negotiation.cc:189] Negotiated authn=JWT
0805 23:10:34.564468 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:34.564473 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:34.565638 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:34.565641 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:34.566341 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:34.566348 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:34.566946 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:34.566950 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:34.567050 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:34.567088 (+ 38us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0805 23:10:34.580553 (+ 13465us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34277/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0805 23:10:34.580723 (+ 170us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.25.92.254:44183: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34277/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":231,"threads_started":1}
I20260805 23:10:34.582046 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 31274
2026-08-05T23:10:34Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (341 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260805 23:10:34.623343 25971 mini_oidc.cc:152] Starting JWKS server
I20260805 23:10:34.624306 25971 webserver.cc:460] Webserver started at https://127.0.0.1:46159/ using document root <none> and password file <none>
I20260805 23:10:34.624435 25971 mini_oidc.cc:189] Starting OIDC Discovery server
I20260805 23:10:34.624755 25971 webserver.cc:460] Webserver started at http://127.0.0.1:43257/ using document root <none> and password file <none>
2026-08-05T23:10:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:10:34Z Disabled control of system clock
I20260805 23:10:34.634908 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785971406251336-25971-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785971406251336-25971-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785971406251336-25971-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.254:46135
--webserver_interface=127.25.92.254
--webserver_port=0
--builtin_ntp_servers=127.25.92.212:32815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.92.254:46135
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46159/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260805 23:10:34.750609 31378 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:10:34.750936 31378 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:10:34.751022 31378 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:10:34.755216 31378 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:10:34.755354 31378 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:10:34.755393 31378 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:10:34.755436 31378 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:10:34.755564 31378 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260805 23:10:34.755606 31378 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260805 23:10:34.755641 31378 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:46159/jwks.json
I20260805 23:10:34.761294 31378 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:32815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785971406251336-25971-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785971406251336-25971-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.92.254:46135
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.254:46135
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46159/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.92.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31378
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785971406251336-25971-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:10:34.762593 31378 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:10:34.764130 31378 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:10:34.773062 31383 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:34.773056 31386 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:34.773059 31384 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:10:34.773564 31378 server_base.cc:1034] running on GCE node
I20260805 23:10:34.774119 31378 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:10:34.775336 31378 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:10:34.776548 31378 hybrid_clock.cc:648] HybridClock initialized: now 1785971434776528 us; error 35 us; skew 500 ppm
I20260805 23:10:34.778548 31378 webserver.cc:460] Webserver started at http://127.25.92.254:45435/ using document root <none> and password file <none>
I20260805 23:10:34.779146 31378 fs_manager.cc:359] Metadata directory not provided
I20260805 23:10:34.779237 31378 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:10:34.779469 31378 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:10:34.781702 31378 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785971406251336-25971-0/minicluster-data/master-0/data/instance:
uuid: "1f2763f3cbfe4f399a67a755e6f2804c"
format_stamp: "Formatted at 2026-08-05 23:10:34 on dist-test-slave-ss4n"
server_key: "9e182461f7cec5eb9f5f80d58453aa86"
server_key_iv: "9229902e95713409c2b756106393aea1"
server_key_version: "encryptionkey@0"
I20260805 23:10:34.782291 31378 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785971406251336-25971-0/minicluster-data/master-0/wal/instance:
uuid: "1f2763f3cbfe4f399a67a755e6f2804c"
format_stamp: "Formatted at 2026-08-05 23:10:34 on dist-test-slave-ss4n"
server_key: "9e182461f7cec5eb9f5f80d58453aa86"
server_key_iv: "9229902e95713409c2b756106393aea1"
server_key_version: "encryptionkey@0"
I20260805 23:10:34.786185 31378 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260805 23:10:34.788693 31392 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:34.790041 31378 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260805 23:10:34.790215 31378 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785971406251336-25971-0/minicluster-data/master-0/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785971406251336-25971-0/minicluster-data/master-0/wal
uuid: "1f2763f3cbfe4f399a67a755e6f2804c"
format_stamp: "Formatted at 2026-08-05 23:10:34 on dist-test-slave-ss4n"
server_key: "9e182461f7cec5eb9f5f80d58453aa86"
server_key_iv: "9229902e95713409c2b756106393aea1"
server_key_version: "encryptionkey@0"
I20260805 23:10:34.790338 31378 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785971406251336-25971-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785971406251336-25971-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785971406251336-25971-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:10:34.811751 31378 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:10:34.812510 31378 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:10:34.812733 31378 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:10:34.820303 31378 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.254:46135
I20260805 23:10:34.820293 31444 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.254:46135 every 8 connection(s)
I20260805 23:10:34.821373 31378 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
I20260805 23:10:34.824667 31446 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:10:34.830363 31446 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f2763f3cbfe4f399a67a755e6f2804c: Bootstrap starting.
I20260805 23:10:34.831717 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 31378
I20260805 23:10:34.831849 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785971406251336-25971-0/minicluster-data/master-0/wal/instance
I20260805 23:10:34.832144 25971 external_mini_cluster.cc:1428] Setting key b4320e4bdde4efc1b575aaffae7980ac
I20260805 23:10:34.833359 31446 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1f2763f3cbfe4f399a67a755e6f2804c: Neither blocks nor log segments found. Creating new log.
I20260805 23:10:34.834185 31446 log.cc:824] T 00000000000000000000000000000000 P 1f2763f3cbfe4f399a67a755e6f2804c: Log is configured to *not* fsync() on all Append() calls
I20260805 23:10:34.836373 31446 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f2763f3cbfe4f399a67a755e6f2804c: No bootstrap required, opened a new log
I20260805 23:10:34.841022 31446 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1f2763f3cbfe4f399a67a755e6f2804c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f2763f3cbfe4f399a67a755e6f2804c" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 46135 } }
I20260805 23:10:34.841322 31446 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1f2763f3cbfe4f399a67a755e6f2804c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:10:34.841401 31446 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1f2763f3cbfe4f399a67a755e6f2804c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f2763f3cbfe4f399a67a755e6f2804c, State: Initialized, Role: FOLLOWER
I20260805 23:10:34.841881 31446 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1f2763f3cbfe4f399a67a755e6f2804c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f2763f3cbfe4f399a67a755e6f2804c" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 46135 } }
I20260805 23:10:34.842038 31446 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1f2763f3cbfe4f399a67a755e6f2804c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:10:34.842124 31446 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1f2763f3cbfe4f399a67a755e6f2804c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:10:34.842252 31446 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1f2763f3cbfe4f399a67a755e6f2804c [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:10:34.842309 31449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:34.833399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32952 (local address 127.25.92.254:46135)
0805 23:10:34.834143 (+ 744us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:34.834160 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:34.834197 (+ 37us) server_negotiation.cc:407] Connection header received
0805 23:10:34.834917 (+ 720us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:34.834934 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:34.835297 (+ 363us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:34.835598 (+ 301us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:34.835944 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:34.836979 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:34.838407 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:34.838998 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:34.839135 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:34.840157 (+ 1022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:34.840161 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:34.840176 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:10:34.840600 (+ 424us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:34.840659 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:10:34.840679 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:34.840756 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:34.840928 (+ 172us) server_negotiation.cc:299] Negotiation successful
0805 23:10:34.841238 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":347,"thread_start_us":155,"threads_started":1}
I20260805 23:10:34.843508 31446 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1f2763f3cbfe4f399a67a755e6f2804c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f2763f3cbfe4f399a67a755e6f2804c" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 46135 } }
I20260805 23:10:34.843869 31446 leader_election.cc:302] T 00000000000000000000000000000000 P 1f2763f3cbfe4f399a67a755e6f2804c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1f2763f3cbfe4f399a67a755e6f2804c; no voters:
I20260805 23:10:34.844225 31446 leader_election.cc:288] T 00000000000000000000000000000000 P 1f2763f3cbfe4f399a67a755e6f2804c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 23:10:34.844326 31451 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1f2763f3cbfe4f399a67a755e6f2804c [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:10:34.844563 31451 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1f2763f3cbfe4f399a67a755e6f2804c [term 1 LEADER]: Becoming Leader. State: Replica: 1f2763f3cbfe4f399a67a755e6f2804c, State: Running, Role: LEADER
I20260805 23:10:34.844887 31451 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1f2763f3cbfe4f399a67a755e6f2804c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f2763f3cbfe4f399a67a755e6f2804c" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 46135 } }
I20260805 23:10:34.845508 25971 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260805 23:10:34.845646 31446 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1f2763f3cbfe4f399a67a755e6f2804c [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:10:34.847087 31452 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f2763f3cbfe4f399a67a755e6f2804c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1f2763f3cbfe4f399a67a755e6f2804c. Latest consensus state: current_term: 1 leader_uuid: "1f2763f3cbfe4f399a67a755e6f2804c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f2763f3cbfe4f399a67a755e6f2804c" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 46135 } } }
I20260805 23:10:34.847208 31452 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f2763f3cbfe4f399a67a755e6f2804c [sys.catalog]: This master's current role is: LEADER
I20260805 23:10:34.847612 31453 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f2763f3cbfe4f399a67a755e6f2804c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1f2763f3cbfe4f399a67a755e6f2804c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f2763f3cbfe4f399a67a755e6f2804c" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 46135 } } }
I20260805 23:10:34.847767 31453 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f2763f3cbfe4f399a67a755e6f2804c [sys.catalog]: This master's current role is: LEADER
W20260805 23:10:34.850476 31467 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1f2763f3cbfe4f399a67a755e6f2804c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260805 23:10:34.850569 31467 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260805 23:10:34.850642 31460 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:10:34.851753 31460 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:10:34.857431 31460 catalog_manager.cc:1347] Generated new cluster ID: b4bd216a6fdf40729f123a7a5a887502
I20260805 23:10:34.857553 31460 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:10:34.868402 31460 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:10:34.869740 31460 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:10:34.878883 31460 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1f2763f3cbfe4f399a67a755e6f2804c: Generated new TSK 0
I20260805 23:10:34.879630 31460 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260805 23:10:34.900312 31449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:34.894890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32962 (local address 127.25.92.254:46135)
0805 23:10:34.895053 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:34.895058 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:34.895230 (+ 172us) server_negotiation.cc:407] Connection header received
0805 23:10:34.895397 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:34.895399 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:34.895449 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:34.895544 (+ 95us) server_negotiation.cc:226] Negotiated authn=JWT
0805 23:10:34.895873 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:34.896441 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:34.897708 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:34.897947 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:34.898037 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:34.898383 (+ 346us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0805 23:10:34.899905 (+ 1522us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46159/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0805 23:10:34.900156 (+ 251us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:32962: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46159/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":62}
W20260805 23:10:34.900386 31449 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:32962: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46159/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260805 23:10:34.902264 31475 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:10:34.894798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:46135 (local address 127.0.0.1:32962)
0805 23:10:34.895133 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:34.895150 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:34.895269 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:34.895592 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:34.895595 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:34.895603 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0805 23:10:34.895728 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:34.895735 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:34.896566 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:34.896570 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:34.897586 (+ 1016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:34.897596 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:34.898076 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:34.898080 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:34.898174 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:34.898203 (+ 29us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0805 23:10:34.900073 (+ 1870us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46159/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0805 23:10:34.902068 (+ 1995us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.25.92.254:46135: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46159/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":231,"mutex_wait_us":1775,"thread_start_us":110,"threads_started":1}
I20260805 23:10:34.902842 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 31378
E20260805 23:10:34.911296 31476 webserver.cc:533] Webserver: sslize failed:
E20260805 23:10:34.911375 31476 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-05T23:10:34Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (319 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-05T23:10:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:10:34Z Disabled control of system clock
I20260805 23:10:34.952720 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785971406251336-25971-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785971406251336-25971-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785971406251336-25971-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.254:35781
--webserver_interface=127.25.92.254
--webserver_port=0
--builtin_ntp_servers=127.25.92.212:40295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.92.254:35781
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785971406251336-25971-0/testchainca.cert with env {}
W20260805 23:10:35.091511 31480 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:10:35.091861 31480 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:10:35.091913 31480 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:10:35.096774 31480 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:10:35.096864 31480 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:10:35.096886 31480 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:10:35.096908 31480 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:10:35.096925 31480 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260805 23:10:35.097000 31480 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260805 23:10:35.097024 31480 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260805 23:10:35.103148 31480 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:40295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785971406251336-25971-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785971406251336-25971-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.92.254:35781
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.254:35781
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.92.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785971406251336-25971-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31480
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785971406251336-25971-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:10:35.104314 31480 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:10:35.105376 31480 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:10:35.114372 31488 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:35.114362 31486 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:35.114358 31485 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:10:35.114591 31480 server_base.cc:1034] running on GCE node
I20260805 23:10:35.115218 31480 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:10:35.116426 31480 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:10:35.117614 31480 hybrid_clock.cc:648] HybridClock initialized: now 1785971435117599 us; error 27 us; skew 500 ppm
I20260805 23:10:35.119742 31480 webserver.cc:460] Webserver started at http://127.25.92.254:44223/ using document root <none> and password file <none>
I20260805 23:10:35.120355 31480 fs_manager.cc:359] Metadata directory not provided
I20260805 23:10:35.120446 31480 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:10:35.120678 31480 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:10:35.122866 31480 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785971406251336-25971-0/minicluster-data/master-0/data/instance:
uuid: "f890e20215bc447fae4077b348592340"
format_stamp: "Formatted at 2026-08-05 23:10:35 on dist-test-slave-ss4n"
server_key: "6ab5964525b8dbdd06fef236cb453fee"
server_key_iv: "6e1d89f7c7e8aa5173f2bcda5d8299e8"
server_key_version: "encryptionkey@0"
I20260805 23:10:35.123466 31480 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785971406251336-25971-0/minicluster-data/master-0/wal/instance:
uuid: "f890e20215bc447fae4077b348592340"
format_stamp: "Formatted at 2026-08-05 23:10:35 on dist-test-slave-ss4n"
server_key: "6ab5964525b8dbdd06fef236cb453fee"
server_key_iv: "6e1d89f7c7e8aa5173f2bcda5d8299e8"
server_key_version: "encryptionkey@0"
I20260805 23:10:35.127400 31480 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260805 23:10:35.130070 31494 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:35.131423 31480 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:10:35.131587 31480 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785971406251336-25971-0/minicluster-data/master-0/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785971406251336-25971-0/minicluster-data/master-0/wal
uuid: "f890e20215bc447fae4077b348592340"
format_stamp: "Formatted at 2026-08-05 23:10:35 on dist-test-slave-ss4n"
server_key: "6ab5964525b8dbdd06fef236cb453fee"
server_key_iv: "6e1d89f7c7e8aa5173f2bcda5d8299e8"
server_key_version: "encryptionkey@0"
I20260805 23:10:35.131716 31480 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785971406251336-25971-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785971406251336-25971-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785971406251336-25971-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:10:35.143332 31480 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:10:35.144202 31480 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:10:35.144415 31480 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:10:35.152069 31480 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.254:35781
I20260805 23:10:35.152104 31546 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.254:35781 every 8 connection(s)
I20260805 23:10:35.153112 31480 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
I20260805 23:10:35.156265 31547 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:10:35.159672 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 31480
I20260805 23:10:35.159813 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785971406251336-25971-0/minicluster-data/master-0/wal/instance
I20260805 23:10:35.160120 25971 external_mini_cluster.cc:1428] Setting key 409fbc6f0f92f1f72cd4d81ce16f15c4
I20260805 23:10:35.163414 31547 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f890e20215bc447fae4077b348592340: Bootstrap starting.
I20260805 23:10:35.165904 31547 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f890e20215bc447fae4077b348592340: Neither blocks nor log segments found. Creating new log.
I20260805 23:10:35.166731 31547 log.cc:824] T 00000000000000000000000000000000 P f890e20215bc447fae4077b348592340: Log is configured to *not* fsync() on all Append() calls
I20260805 23:10:35.169219 31547 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f890e20215bc447fae4077b348592340: No bootstrap required, opened a new log
I20260805 23:10:35.171861 31550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:35.161830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44422 (local address 127.25.92.254:35781)
0805 23:10:35.162674 (+ 844us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:35.162691 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:35.162732 (+ 41us) server_negotiation.cc:407] Connection header received
0805 23:10:35.163470 (+ 738us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:35.163530 (+ 60us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:35.163979 (+ 449us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:35.164334 (+ 355us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:35.164822 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:35.166025 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:35.167072 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:35.167833 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:35.168012 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:35.169051 (+ 1039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:35.169058 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:35.169077 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:10:35.169676 (+ 599us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:35.169762 (+ 86us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:10:35.169796 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:35.169906 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:35.170541 (+ 635us) server_negotiation.cc:299] Negotiation successful
0805 23:10:35.170865 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":413,"thread_start_us":188,"threads_started":1}
I20260805 23:10:35.173870 31547 raft_consensus.cc:348] T 00000000000000000000000000000000 P f890e20215bc447fae4077b348592340 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f890e20215bc447fae4077b348592340" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 35781 } }
I20260805 23:10:35.174175 31547 raft_consensus.cc:374] T 00000000000000000000000000000000 P f890e20215bc447fae4077b348592340 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:10:35.174238 31547 raft_consensus.cc:729] T 00000000000000000000000000000000 P f890e20215bc447fae4077b348592340 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f890e20215bc447fae4077b348592340, State: Initialized, Role: FOLLOWER
I20260805 23:10:35.174708 31547 consensus_queue.cc:260] T 00000000000000000000000000000000 P f890e20215bc447fae4077b348592340 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f890e20215bc447fae4077b348592340" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 35781 } }
I20260805 23:10:35.174863 31547 raft_consensus.cc:388] T 00000000000000000000000000000000 P f890e20215bc447fae4077b348592340 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:10:35.174937 31547 raft_consensus.cc:482] T 00000000000000000000000000000000 P f890e20215bc447fae4077b348592340 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:10:35.175066 31547 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f890e20215bc447fae4077b348592340 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:10:35.175091 25971 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260805 23:10:35.176326 31547 raft_consensus.cc:504] T 00000000000000000000000000000000 P f890e20215bc447fae4077b348592340 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f890e20215bc447fae4077b348592340" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 35781 } }
I20260805 23:10:35.176692 31547 leader_election.cc:302] T 00000000000000000000000000000000 P f890e20215bc447fae4077b348592340 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f890e20215bc447fae4077b348592340; no voters:
I20260805 23:10:35.177070 31547 leader_election.cc:288] T 00000000000000000000000000000000 P f890e20215bc447fae4077b348592340 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 23:10:35.177140 31552 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f890e20215bc447fae4077b348592340 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:10:35.177336 31552 raft_consensus.cc:686] T 00000000000000000000000000000000 P f890e20215bc447fae4077b348592340 [term 1 LEADER]: Becoming Leader. State: Replica: f890e20215bc447fae4077b348592340, State: Running, Role: LEADER
I20260805 23:10:35.177726 31552 consensus_queue.cc:237] T 00000000000000000000000000000000 P f890e20215bc447fae4077b348592340 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f890e20215bc447fae4077b348592340" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 35781 } }
I20260805 23:10:35.178489 31547 sys_catalog.cc:564] T 00000000000000000000000000000000 P f890e20215bc447fae4077b348592340 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:10:35.179533 31553 sys_catalog.cc:455] T 00000000000000000000000000000000 P f890e20215bc447fae4077b348592340 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f890e20215bc447fae4077b348592340. Latest consensus state: current_term: 1 leader_uuid: "f890e20215bc447fae4077b348592340" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f890e20215bc447fae4077b348592340" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 35781 } } }
I20260805 23:10:35.179700 31553 sys_catalog.cc:458] T 00000000000000000000000000000000 P f890e20215bc447fae4077b348592340 [sys.catalog]: This master's current role is: LEADER
I20260805 23:10:35.180629 31554 sys_catalog.cc:455] T 00000000000000000000000000000000 P f890e20215bc447fae4077b348592340 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f890e20215bc447fae4077b348592340" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f890e20215bc447fae4077b348592340" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 35781 } } }
I20260805 23:10:35.180757 31554 sys_catalog.cc:458] T 00000000000000000000000000000000 P f890e20215bc447fae4077b348592340 [sys.catalog]: This master's current role is: LEADER
W20260805 23:10:35.183499 31567 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f890e20215bc447fae4077b348592340: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260805 23:10:35.183578 31567 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260805 23:10:35.183665 31561 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:10:35.184485 31561 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:10:35.190878 31561 catalog_manager.cc:1347] Generated new cluster ID: acf11ae7dca140d98078a6698a1d7fdf
I20260805 23:10:35.190986 31561 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:10:35.216065 31561 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:10:35.217015 31561 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:10:35.232367 31561 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f890e20215bc447fae4077b348592340: Generated new TSK 0
I20260805 23:10:35.233157 31561 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260805 23:10:35.713790 31550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:35.248251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44436 (local address 127.25.92.254:35781)
0805 23:10:35.248392 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:35.248399 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:35.248572 (+ 173us) server_negotiation.cc:407] Connection header received
0805 23:10:35.248767 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:35.248770 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:35.248819 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:35.248930 (+ 111us) server_negotiation.cc:226] Negotiated authn=JWT
0805 23:10:35.249187 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:35.249786 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:35.250688 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:35.250918 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:35.251020 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:35.251359 (+ 339us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0805 23:10:35.713211 (+461852us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0805 23:10:35.713621 (+ 410us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:44436: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":55}
W20260805 23:10:35.713896 31550 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:44436: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260805 23:10:35.713922 31576 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:10:35.248101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:35781 (local address 127.0.0.1:44436)
0805 23:10:35.248473 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:35.248492 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:35.248621 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:35.248938 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:35.248941 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:35.248949 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0805 23:10:35.249072 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:35.249083 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:35.249917 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:35.249923 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:35.250564 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:35.250571 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:35.251054 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:35.251057 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:35.251145 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:35.251174 (+ 29us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0805 23:10:35.713507 (+462333us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0805 23:10:35.713722 (+ 215us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.25.92.254:35781: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":107,"threads_started":1}
I20260805 23:10:35.714604 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 31480
2026-08-05T23:10:35Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (791 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (132 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (129 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971406251336-25971-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 23:10:36 dist-test-slave-ss4n krb5kdc[31588](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 23:10:36 dist-test-slave-ss4n krb5kdc[31588](info): set up 2 sockets
Aug 05 23:10:36 dist-test-slave-ss4n krb5kdc[31588](info): commencing operation
krb5kdc: starting...
W20260805 23:10:37.103878 25971 mini_kdc.cc:121] Time spent starting KDC: real 1.106s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971406251336-25971-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971406251336-25971-0/krb5kdc/test-user.keytab.
Aug 05 23:10:37 dist-test-slave-ss4n krb5kdc[31588](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971437, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T23:10:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:10:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.92.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.92.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.92.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab.
Entry for principal kudu/127.25.92.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab.
I20260805 23:10:37.229344 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971406251336-25971-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971406251336-25971-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971406251336-25971-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.254:44049
--webserver_interface=127.25.92.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab
--principal=kudu/127.25.92.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:42559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.92.254:44049
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:10:37.351755 31601 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:10:37.352124 31601 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:10:37.352210 31601 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:10:37.356457 31601 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:10:37.356565 31601 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:10:37.356601 31601 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:10:37.356643 31601 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:10:37.356676 31601 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:10:37.362285 31601 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:42559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971406251336-25971-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971406251336-25971-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.92.254:44049
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab
--principal=kudu/127.25.92.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.254:44049
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.92.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31601
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971406251336-25971-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:10:37.363581 31601 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:10:37.364630 31601 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260805 23:10:37.373668 31601 server_base.cc:1034] running on GCE node
W20260805 23:10:37.373636 31607 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:37.373626 31606 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:37.373858 31609 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:10:37.374572 31601 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:10:37.375771 31601 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:10:37.376956 31601 hybrid_clock.cc:648] HybridClock initialized: now 1785971437376936 us; error 33 us; skew 500 ppm
Aug 05 23:10:37 dist-test-slave-ss4n krb5kdc[31588](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971437, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:10:37.380071 31601 init.cc:377] Logged in from keytab as kudu/127.25.92.254@KRBTEST.COM (short username kudu)
I20260805 23:10:37.381114 31601 webserver.cc:460] Webserver started at http://127.25.92.254:38727/ using document root <none> and password file <none>
I20260805 23:10:37.381793 31601 fs_manager.cc:359] Metadata directory not provided
I20260805 23:10:37.381891 31601 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:10:37.382144 31601 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:10:37.384514 31601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971406251336-25971-0/minicluster-data/master-0/data/instance:
uuid: "2bb796af203a4af785498473f20213aa"
format_stamp: "Formatted at 2026-08-05 23:10:37 on dist-test-slave-ss4n"
server_key: "a752adc7eea01aa46284c8637445d364"
server_key_iv: "2c6833c64b02caaa384f2576dd7e5d00"
server_key_version: "encryptionkey@0"
I20260805 23:10:37.385143 31601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971406251336-25971-0/minicluster-data/master-0/wal/instance:
uuid: "2bb796af203a4af785498473f20213aa"
format_stamp: "Formatted at 2026-08-05 23:10:37 on dist-test-slave-ss4n"
server_key: "a752adc7eea01aa46284c8637445d364"
server_key_iv: "2c6833c64b02caaa384f2576dd7e5d00"
server_key_version: "encryptionkey@0"
I20260805 23:10:37.389369 31601 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260805 23:10:37.392009 31616 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:37.393352 31601 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260805 23:10:37.393605 31601 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971406251336-25971-0/minicluster-data/master-0/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971406251336-25971-0/minicluster-data/master-0/wal
uuid: "2bb796af203a4af785498473f20213aa"
format_stamp: "Formatted at 2026-08-05 23:10:37 on dist-test-slave-ss4n"
server_key: "a752adc7eea01aa46284c8637445d364"
server_key_iv: "2c6833c64b02caaa384f2576dd7e5d00"
server_key_version: "encryptionkey@0"
I20260805 23:10:37.393744 31601 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971406251336-25971-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971406251336-25971-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971406251336-25971-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:10:37.409243 31601 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:10:37.412366 31601 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:10:37.412613 31601 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:10:37.420085 31601 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.254:44049
I20260805 23:10:37.420111 31668 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.254:44049 every 8 connection(s)
I20260805 23:10:37.421056 31601 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
I20260805 23:10:37.423909 31669 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:10:37.426888 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 31601
I20260805 23:10:37.427060 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971406251336-25971-0/minicluster-data/master-0/wal/instance
I20260805 23:10:37.427333 25971 external_mini_cluster.cc:1428] Setting key 8d7887edc48a308e48aee2495e6ff94e
I20260805 23:10:37.430125 31669 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2bb796af203a4af785498473f20213aa: Bootstrap starting.
I20260805 23:10:37.432490 31669 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2bb796af203a4af785498473f20213aa: Neither blocks nor log segments found. Creating new log.
I20260805 23:10:37.433259 31669 log.cc:824] T 00000000000000000000000000000000 P 2bb796af203a4af785498473f20213aa: Log is configured to *not* fsync() on all Append() calls
I20260805 23:10:37.435277 31669 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2bb796af203a4af785498473f20213aa: No bootstrap required, opened a new log
Aug 05 23:10:37 dist-test-slave-ss4n krb5kdc[31588](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971437, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM
I20260805 23:10:37.439682 31669 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2bb796af203a4af785498473f20213aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bb796af203a4af785498473f20213aa" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 44049 } }
I20260805 23:10:37.439985 31669 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2bb796af203a4af785498473f20213aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:10:37.440060 31669 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2bb796af203a4af785498473f20213aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2bb796af203a4af785498473f20213aa, State: Initialized, Role: FOLLOWER
I20260805 23:10:37.440557 31669 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2bb796af203a4af785498473f20213aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bb796af203a4af785498473f20213aa" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 44049 } }
I20260805 23:10:37.440737 31669 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2bb796af203a4af785498473f20213aa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:10:37.440811 31669 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2bb796af203a4af785498473f20213aa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:10:37.440910 31669 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2bb796af203a4af785498473f20213aa [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:10:37.442219 31669 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2bb796af203a4af785498473f20213aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bb796af203a4af785498473f20213aa" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 44049 } }
I20260805 23:10:37.442597 31669 leader_election.cc:302] T 00000000000000000000000000000000 P 2bb796af203a4af785498473f20213aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2bb796af203a4af785498473f20213aa; no voters:
I20260805 23:10:37.442977 31669 leader_election.cc:288] T 00000000000000000000000000000000 P 2bb796af203a4af785498473f20213aa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 23:10:37.443110 31674 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2bb796af203a4af785498473f20213aa [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:10:37.443364 31674 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2bb796af203a4af785498473f20213aa [term 1 LEADER]: Becoming Leader. State: Replica: 2bb796af203a4af785498473f20213aa, State: Running, Role: LEADER
I20260805 23:10:37.443418 31672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:37.428914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55518 (local address 127.25.92.254:44049)
0805 23:10:37.429430 (+ 516us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:37.429446 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:37.429546 (+ 100us) server_negotiation.cc:407] Connection header received
0805 23:10:37.430345 (+ 799us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:37.430366 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:37.430733 (+ 367us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:37.431108 (+ 375us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:37.432071 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:37.433231 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:37.434250 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:37.434769 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:37.434947 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:37.437737 (+ 2790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:37.437745 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:37.437768 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:37.437811 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:37.439880 (+ 2069us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:37.440437 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:37.440442 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:37.440449 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:37.440616 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:37.440976 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:37.440979 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:37.440982 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:37.441541 (+ 559us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:37.441753 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:37.442041 (+ 288us) server_negotiation.cc:299] Negotiation successful
0805 23:10:37.442390 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":162,"threads_started":1}
I20260805 23:10:37.443724 31674 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2bb796af203a4af785498473f20213aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bb796af203a4af785498473f20213aa" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 44049 } }
I20260805 23:10:37.444279 31669 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2bb796af203a4af785498473f20213aa [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:10:37.445346 31675 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2bb796af203a4af785498473f20213aa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2bb796af203a4af785498473f20213aa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bb796af203a4af785498473f20213aa" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 44049 } } }
I20260805 23:10:37.445463 31675 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2bb796af203a4af785498473f20213aa [sys.catalog]: This master's current role is: LEADER
I20260805 23:10:37.445833 31676 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2bb796af203a4af785498473f20213aa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2bb796af203a4af785498473f20213aa. Latest consensus state: current_term: 1 leader_uuid: "2bb796af203a4af785498473f20213aa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bb796af203a4af785498473f20213aa" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 44049 } } }
I20260805 23:10:37.445912 31676 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2bb796af203a4af785498473f20213aa [sys.catalog]: This master's current role is: LEADER
W20260805 23:10:37.449173 31690 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2bb796af203a4af785498473f20213aa: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260805 23:10:37.449266 31690 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260805 23:10:37.449347 31683 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:10:37.450274 31683 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:10:37.456113 31683 catalog_manager.cc:1347] Generated new cluster ID: 7e9ace199bb94189ab552b0e9b10b7db
I20260805 23:10:37.456202 31683 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:10:37.482654 31683 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:10:37.483683 31683 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:10:37.490190 31683 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2bb796af203a4af785498473f20213aa: Generated new TSK 0
I20260805 23:10:37.490895 31683 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.92.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.92.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.92.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.193.keytab.
Entry for principal kudu/127.25.92.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.193.keytab.
I20260805 23:10:37.525246 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971406251336-25971-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971406251336-25971-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971406251336-25971-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971406251336-25971-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.92.193:0
--local_ip_for_outbound_sockets=127.25.92.193
--webserver_interface=127.25.92.193
--webserver_port=0
--tserver_master_addrs=127.25.92.254:44049
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.193.keytab
--principal=kudu/127.25.92.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:42559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:10:37.649617 31697 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:10:37.649967 31697 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:10:37.650053 31697 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:10:37.654595 31697 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:10:37.654692 31697 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:10:37.654788 31697 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.92.193
I20260805 23:10:37.660281 31697 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:42559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971406251336-25971-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971406251336-25971-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.193.keytab
--principal=kudu/127.25.92.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.92.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971406251336-25971-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.92.193
--webserver_port=0
--tserver_master_addrs=127.25.92.254:44049
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31697
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.92.193
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971406251336-25971-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:10:37.661530 31697 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:10:37.662633 31697 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:10:37.672818 31703 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:37.672811 31704 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:37.672811 31706 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:10:37.673120 31697 server_base.cc:1034] running on GCE node
I20260805 23:10:37.673597 31697 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:10:37.674244 31697 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:10:37.675415 31697 hybrid_clock.cc:648] HybridClock initialized: now 1785971437675394 us; error 34 us; skew 500 ppm
Aug 05 23:10:37 dist-test-slave-ss4n krb5kdc[31588](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971437, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:10:37.678293 31697 init.cc:377] Logged in from keytab as kudu/127.25.92.193@KRBTEST.COM (short username kudu)
I20260805 23:10:37.679253 31697 webserver.cc:460] Webserver started at http://127.25.92.193:42277/ using document root <none> and password file <none>
I20260805 23:10:37.679844 31697 fs_manager.cc:359] Metadata directory not provided
I20260805 23:10:37.679908 31697 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:10:37.680119 31697 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:10:37.682334 31697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971406251336-25971-0/minicluster-data/ts-0/data/instance:
uuid: "e9cc7c8dca724e39895e407e329ce464"
format_stamp: "Formatted at 2026-08-05 23:10:37 on dist-test-slave-ss4n"
server_key: "7f6c2f4d1e8155c8c50fc50e385eb80f"
server_key_iv: "3bd2d022bd45693e924c68439f6546c5"
server_key_version: "encryptionkey@0"
I20260805 23:10:37.682911 31697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971406251336-25971-0/minicluster-data/ts-0/wal/instance:
uuid: "e9cc7c8dca724e39895e407e329ce464"
format_stamp: "Formatted at 2026-08-05 23:10:37 on dist-test-slave-ss4n"
server_key: "7f6c2f4d1e8155c8c50fc50e385eb80f"
server_key_iv: "3bd2d022bd45693e924c68439f6546c5"
server_key_version: "encryptionkey@0"
I20260805 23:10:37.686717 31697 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260805 23:10:37.689163 31713 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:37.690508 31697 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260805 23:10:37.690639 31697 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971406251336-25971-0/minicluster-data/ts-0/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971406251336-25971-0/minicluster-data/ts-0/wal
uuid: "e9cc7c8dca724e39895e407e329ce464"
format_stamp: "Formatted at 2026-08-05 23:10:37 on dist-test-slave-ss4n"
server_key: "7f6c2f4d1e8155c8c50fc50e385eb80f"
server_key_iv: "3bd2d022bd45693e924c68439f6546c5"
server_key_version: "encryptionkey@0"
I20260805 23:10:37.690759 31697 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971406251336-25971-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971406251336-25971-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971406251336-25971-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:10:37.713833 31697 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:10:37.717133 31697 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:10:37.717336 31697 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:10:37.718003 31697 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:10:37.718858 31697 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:10:37.718909 31697 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:37.718948 31697 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:10:37.718964 31697 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:37.728468 31697 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.193:46425
I20260805 23:10:37.728554 31830 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.193:46425 every 8 connection(s)
I20260805 23:10:37.729468 31697 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971406251336-25971-0/minicluster-data/ts-0/data/info.pb
I20260805 23:10:37.732883 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 31697
I20260805 23:10:37.733009 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971406251336-25971-0/minicluster-data/ts-0/wal/instance
I20260805 23:10:37.733345 25971 external_mini_cluster.cc:1428] Setting key 5546056734ab7fe2ef25ef2412749225
Aug 05 23:10:37 dist-test-slave-ss4n krb5kdc[31588](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971437, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.193@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.25.92.194@KRBTEST.COM; defaulting to no policy
I20260805 23:10:37.745160 31672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:37.731505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.193:41057 (local address 127.25.92.254:44049)
0805 23:10:37.731677 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:37.731683 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:37.732670 (+ 987us) server_negotiation.cc:407] Connection header received
0805 23:10:37.734417 (+ 1747us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:37.734421 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:37.734477 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:37.734582 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:37.736101 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:37.736733 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:37.738117 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:37.738371 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:37.738494 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:37.741212 (+ 2718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:37.741216 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:37.741219 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:37.741251 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:37.743000 (+ 1749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:37.743581 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:37.743584 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:37.743585 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:37.743671 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:37.744016 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:37.744018 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:37.744019 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:37.744222 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:37.744301 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:37.744923 (+ 622us) server_negotiation.cc:299] Negotiation successful
0805 23:10:37.745035 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260805 23:10:37.746171 31833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:37.731999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:44049 (local address 127.25.92.193:41057)
0805 23:10:37.732478 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:37.732524 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:37.734203 (+ 1679us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:37.734763 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:37.734769 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:37.735233 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:37.735923 (+ 690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:37.735945 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:37.736878 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:37.736883 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:37.737956 (+ 1073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:37.737967 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:37.738540 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:37.738544 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:37.738777 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:37.739441 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:10:37.739458 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:37.741017 (+ 1559us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:10:37.743135 (+ 2118us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:37.743141 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:37.743155 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:37.743482 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:37.743781 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:37.743784 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:37.743785 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:37.743927 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:37.744315 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:10:37.744320 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:10:37.744663 (+ 343us) client_negotiation.cc:769] Sending connection context
0805 23:10:37.744888 (+ 225us) client_negotiation.cc:240] Negotiation successful
0805 23:10:37.745159 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":120,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.92.194@KRBTEST.COM" created.
I20260805 23:10:37.747527 31831 heartbeater.cc:347] Connected to a master server at 127.25.92.254:44049
I20260805 23:10:37.747794 31831 heartbeater.cc:464] Registering TS with master...
I20260805 23:10:37.748292 31831 heartbeater.cc:511] Master 127.25.92.254:44049 requested a full tablet report, sending...
I20260805 23:10:37.749945 31633 ts_manager.cc:194] Registered new tserver with Master: e9cc7c8dca724e39895e407e329ce464 (127.25.92.193:46425)
I20260805 23:10:37.751657 31633 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.92.193@KRBTEST.COM'} at 127.25.92.193:41057
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.92.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.194.keytab.
Entry for principal kudu/127.25.92.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.194.keytab.
I20260805 23:10:37.765898 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971406251336-25971-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971406251336-25971-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971406251336-25971-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971406251336-25971-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.92.194:0
--local_ip_for_outbound_sockets=127.25.92.194
--webserver_interface=127.25.92.194
--webserver_port=0
--tserver_master_addrs=127.25.92.254:44049
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.194.keytab
--principal=kudu/127.25.92.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:42559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:10:37.878738 31836 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:10:37.879029 31836 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:10:37.879074 31836 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:10:37.883309 31836 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:10:37.883380 31836 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:10:37.883471 31836 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.92.194
I20260805 23:10:37.889026 31836 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:42559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971406251336-25971-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971406251336-25971-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.194.keytab
--principal=kudu/127.25.92.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.92.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971406251336-25971-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.92.194
--webserver_port=0
--tserver_master_addrs=127.25.92.254:44049
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31836
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.92.194
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971406251336-25971-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:10:37.890161 31836 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:10:37.891208 31836 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:10:37.901567 31844 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:37.901623 31842 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:37.901603 31841 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:10:37.901865 31836 server_base.cc:1034] running on GCE node
I20260805 23:10:37.902367 31836 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:10:37.903168 31836 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:10:37.904373 31836 hybrid_clock.cc:648] HybridClock initialized: now 1785971437904340 us; error 48 us; skew 500 ppm
Aug 05 23:10:37 dist-test-slave-ss4n krb5kdc[31588](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971437, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:10:37.907331 31836 init.cc:377] Logged in from keytab as kudu/127.25.92.194@KRBTEST.COM (short username kudu)
I20260805 23:10:37.908228 31836 webserver.cc:460] Webserver started at http://127.25.92.194:37197/ using document root <none> and password file <none>
I20260805 23:10:37.908758 31836 fs_manager.cc:359] Metadata directory not provided
I20260805 23:10:37.908844 31836 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:10:37.909081 31836 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:10:37.911203 31836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971406251336-25971-0/minicluster-data/ts-1/data/instance:
uuid: "94d6841283bc4e8f977e675ef7eaa40d"
format_stamp: "Formatted at 2026-08-05 23:10:37 on dist-test-slave-ss4n"
server_key: "61dbc5516979ceddf22643cbaf608ce1"
server_key_iv: "400e65bd66f6ce6fcd9f03f77d0abd0c"
server_key_version: "encryptionkey@0"
I20260805 23:10:37.911720 31836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971406251336-25971-0/minicluster-data/ts-1/wal/instance:
uuid: "94d6841283bc4e8f977e675ef7eaa40d"
format_stamp: "Formatted at 2026-08-05 23:10:37 on dist-test-slave-ss4n"
server_key: "61dbc5516979ceddf22643cbaf608ce1"
server_key_iv: "400e65bd66f6ce6fcd9f03f77d0abd0c"
server_key_version: "encryptionkey@0"
I20260805 23:10:37.916286 31836 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260805 23:10:37.919209 31851 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:37.920804 31836 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260805 23:10:37.920948 31836 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971406251336-25971-0/minicluster-data/ts-1/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971406251336-25971-0/minicluster-data/ts-1/wal
uuid: "94d6841283bc4e8f977e675ef7eaa40d"
format_stamp: "Formatted at 2026-08-05 23:10:37 on dist-test-slave-ss4n"
server_key: "61dbc5516979ceddf22643cbaf608ce1"
server_key_iv: "400e65bd66f6ce6fcd9f03f77d0abd0c"
server_key_version: "encryptionkey@0"
I20260805 23:10:37.921101 31836 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971406251336-25971-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971406251336-25971-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971406251336-25971-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:10:37.939309 31836 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:10:37.942423 31836 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:10:37.942657 31836 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:10:37.943265 31836 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:10:37.944131 31836 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:10:37.944178 31836 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:37.944289 31836 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:10:37.944326 31836 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:37.953780 31836 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.194:35043
I20260805 23:10:37.953797 31969 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.194:35043 every 8 connection(s)
I20260805 23:10:37.954725 31836 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971406251336-25971-0/minicluster-data/ts-1/data/info.pb
I20260805 23:10:37.963001 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 31836
I20260805 23:10:37.963109 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971406251336-25971-0/minicluster-data/ts-1/wal/instance
I20260805 23:10:37.963321 25971 external_mini_cluster.cc:1428] Setting key 4bf1ef7b4353e4f7d80c69e1854aa6cb
Aug 05 23:10:37 dist-test-slave-ss4n krb5kdc[31588](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971437, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.194@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM
I20260805 23:10:37.969244 31672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:37.956490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.194:39493 (local address 127.25.92.254:44049)
0805 23:10:37.956622 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:37.956626 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:37.957425 (+ 799us) server_negotiation.cc:407] Connection header received
0805 23:10:37.958864 (+ 1439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:37.958868 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:37.958932 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:37.959028 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:37.960199 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:37.960745 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:37.961976 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:37.962219 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:37.962327 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:37.965042 (+ 2715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:37.965046 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:37.965048 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:37.965078 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:37.966902 (+ 1824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:37.967565 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:37.967567 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:37.967568 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:37.967644 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:37.968064 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:37.968066 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:37.968067 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:37.968256 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:37.968335 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:37.969008 (+ 673us) server_negotiation.cc:299] Negotiation successful
0805 23:10:37.969116 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260805 23:10:37.970628 31972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:37.956813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:44049 (local address 127.25.92.194:39493)
0805 23:10:37.957272 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:37.957315 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:37.958629 (+ 1314us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:37.959207 (+ 578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:37.959213 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:37.959570 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:37.960043 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:37.960056 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:37.960879 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:37.960883 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:37.961848 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:37.961858 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:37.962383 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:37.962387 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:37.962551 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:37.963258 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:10:37.963274 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:37.964880 (+ 1606us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:10:37.967026 (+ 2146us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:37.967033 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:37.967068 (+ 35us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:37.967460 (+ 392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:37.967760 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:37.967765 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:37.967769 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:37.967962 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:37.968363 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:10:37.968398 (+ 35us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:10:37.968746 (+ 348us) client_negotiation.cc:769] Sending connection context
0805 23:10:37.968968 (+ 222us) client_negotiation.cc:240] Negotiation successful
0805 23:10:37.969368 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":120,"threads_started":1}
I20260805 23:10:37.971894 31970 heartbeater.cc:347] Connected to a master server at 127.25.92.254:44049
I20260805 23:10:37.972141 31970 heartbeater.cc:464] Registering TS with master...
I20260805 23:10:37.972644 31970 heartbeater.cc:511] Master 127.25.92.254:44049 requested a full tablet report, sending...
I20260805 23:10:37.973752 31633 ts_manager.cc:194] Registered new tserver with Master: 94d6841283bc4e8f977e675ef7eaa40d (127.25.92.194:35043)
WARNING: no policy specified for kudu/127.25.92.195@KRBTEST.COM; defaulting to no policy
I20260805 23:10:37.974390 31633 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.92.194@KRBTEST.COM'} at 127.25.92.194:39493
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.92.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.92.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.195.keytab.
Entry for principal kudu/127.25.92.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.195.keytab.
I20260805 23:10:37.993263 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971406251336-25971-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971406251336-25971-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971406251336-25971-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971406251336-25971-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.92.195:0
--local_ip_for_outbound_sockets=127.25.92.195
--webserver_interface=127.25.92.195
--webserver_port=0
--tserver_master_addrs=127.25.92.254:44049
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.195.keytab
--principal=kudu/127.25.92.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:42559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:10:38.104260 31975 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:10:38.104522 31975 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:10:38.104596 31975 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:10:38.108647 31975 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:10:38.108709 31975 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:10:38.108815 31975 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.92.195
I20260805 23:10:38.114212 31975 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:42559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971406251336-25971-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971406251336-25971-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.195.keytab
--principal=kudu/127.25.92.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.92.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971406251336-25971-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.92.195
--webserver_port=0
--tserver_master_addrs=127.25.92.254:44049
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31975
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.92.195
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971406251336-25971-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:10:38.115265 31975 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:10:38.116277 31975 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:10:38.125864 31983 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:38.125833 31981 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:10:38.125970 31975 server_base.cc:1034] running on GCE node
W20260805 23:10:38.125845 31980 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:10:38.126525 31975 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:10:38.127122 31975 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:10:38.128298 31975 hybrid_clock.cc:648] HybridClock initialized: now 1785971438128265 us; error 45 us; skew 500 ppm
Aug 05 23:10:38 dist-test-slave-ss4n krb5kdc[31588](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971438, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:10:38.131093 31975 init.cc:377] Logged in from keytab as kudu/127.25.92.195@KRBTEST.COM (short username kudu)
I20260805 23:10:38.131953 31975 webserver.cc:460] Webserver started at http://127.25.92.195:33459/ using document root <none> and password file <none>
I20260805 23:10:38.132511 31975 fs_manager.cc:359] Metadata directory not provided
I20260805 23:10:38.132598 31975 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:10:38.132836 31975 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:10:38.134989 31975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971406251336-25971-0/minicluster-data/ts-2/data/instance:
uuid: "3bb8c3f7ef914c68a4cbd3c5d1be6ebb"
format_stamp: "Formatted at 2026-08-05 23:10:38 on dist-test-slave-ss4n"
server_key: "5346afd664c28e89ac6ef598809125d1"
server_key_iv: "fa9f030735500088f49d7e37c274bd8a"
server_key_version: "encryptionkey@0"
I20260805 23:10:38.135556 31975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971406251336-25971-0/minicluster-data/ts-2/wal/instance:
uuid: "3bb8c3f7ef914c68a4cbd3c5d1be6ebb"
format_stamp: "Formatted at 2026-08-05 23:10:38 on dist-test-slave-ss4n"
server_key: "5346afd664c28e89ac6ef598809125d1"
server_key_iv: "fa9f030735500088f49d7e37c274bd8a"
server_key_version: "encryptionkey@0"
I20260805 23:10:38.139420 31975 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260805 23:10:38.141883 31990 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:38.143131 31975 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260805 23:10:38.143271 31975 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971406251336-25971-0/minicluster-data/ts-2/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971406251336-25971-0/minicluster-data/ts-2/wal
uuid: "3bb8c3f7ef914c68a4cbd3c5d1be6ebb"
format_stamp: "Formatted at 2026-08-05 23:10:38 on dist-test-slave-ss4n"
server_key: "5346afd664c28e89ac6ef598809125d1"
server_key_iv: "fa9f030735500088f49d7e37c274bd8a"
server_key_version: "encryptionkey@0"
I20260805 23:10:38.143384 31975 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971406251336-25971-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971406251336-25971-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971406251336-25971-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:10:38.154085 31975 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:10:38.157104 31975 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:10:38.157307 31975 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:10:38.157954 31975 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:10:38.158785 31975 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:10:38.158851 31975 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:38.158921 31975 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:10:38.158960 31975 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:38.168030 31975 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.195:33125
I20260805 23:10:38.168046 32106 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.195:33125 every 8 connection(s)
I20260805 23:10:38.168926 31975 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971406251336-25971-0/minicluster-data/ts-2/data/info.pb
I20260805 23:10:38.169778 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 31975
I20260805 23:10:38.169902 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971406251336-25971-0/minicluster-data/ts-2/wal/instance
I20260805 23:10:38.170197 25971 external_mini_cluster.cc:1428] Setting key 796c85fc4ee8a4a38644dfb2aabb0ffb
Aug 05 23:10:38 dist-test-slave-ss4n krb5kdc[31588](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971438, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.195@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM
I20260805 23:10:38.183146 31672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:38.170925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.195:42373 (local address 127.25.92.254:44049)
0805 23:10:38.171079 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:38.171084 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:38.171984 (+ 900us) server_negotiation.cc:407] Connection header received
0805 23:10:38.173387 (+ 1403us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:38.173390 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:38.173435 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:38.173616 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:38.174750 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:38.175280 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:38.176334 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:38.176536 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:38.176654 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:38.178995 (+ 2341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:38.178998 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:38.179000 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:38.179025 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:38.180625 (+ 1600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:38.181217 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:38.181221 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:38.181223 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:38.181331 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:38.181788 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:38.181791 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:38.181792 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:38.182040 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:38.182152 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:38.182821 (+ 669us) server_negotiation.cc:299] Negotiation successful
0805 23:10:38.182985 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260805 23:10:38.184288 32109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:38.171360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:44049 (local address 127.25.92.195:42373)
0805 23:10:38.171832 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:38.171884 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:38.173205 (+ 1321us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:38.173767 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:38.173774 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:38.174122 (+ 348us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:38.174614 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:38.174627 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:38.175384 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:38.175386 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:38.176233 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:38.176242 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:38.176695 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:38.176698 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:38.176829 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:38.177370 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:10:38.177385 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:38.178824 (+ 1439us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:10:38.180757 (+ 1933us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:38.180764 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:38.180779 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:38.181118 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:38.181440 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:38.181444 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:38.181446 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:38.181678 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:38.182148 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:10:38.182154 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:10:38.182470 (+ 316us) client_negotiation.cc:769] Sending connection context
0805 23:10:38.182742 (+ 272us) client_negotiation.cc:240] Negotiation successful
0805 23:10:38.183414 (+ 672us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":134,"threads_started":1}
I20260805 23:10:38.185369 32107 heartbeater.cc:347] Connected to a master server at 127.25.92.254:44049
I20260805 23:10:38.185812 32107 heartbeater.cc:464] Registering TS with master...
I20260805 23:10:38.186288 32107 heartbeater.cc:511] Master 127.25.92.254:44049 requested a full tablet report, sending...
I20260805 23:10:38.187247 31633 ts_manager.cc:194] Registered new tserver with Master: 3bb8c3f7ef914c68a4cbd3c5d1be6ebb (127.25.92.195:33125)
I20260805 23:10:38.187781 31633 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.92.195@KRBTEST.COM'} at 127.25.92.195:42373
I20260805 23:10:38.196627 25971 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971406251336-25971-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971406251336-25971-0/krb5kdc/test-admin.keytab.
Aug 05 23:10:38 dist-test-slave-ss4n krb5kdc[31588](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971438, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:10:38.215216 25971 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260805 23:10:38.218446 32114 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:10:38.216705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36026 (local address 127.25.92.193:46425)
0805 23:10:38.217014 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:38.217030 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:38.217056 (+ 26us) server_negotiation.cc:407] Connection header received
0805 23:10:38.217166 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:38.217171 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:38.217397 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:38.217539 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:38.218235 (+ 696us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36026: BlockingRecv error: recv got EOF from 127.0.0.1:36026 (error 108)
Metrics: {"server-negotiator.queue_time_us":213,"thread_start_us":104,"threads_started":1}
W20260805 23:10:38.218547 25971 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971406251336-25971-0/krb5kdc/krb5cc' of type FILE
Aug 05 23:10:38 dist-test-slave-ss4n krb5kdc[31588](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971438, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:10:38.219614 25971 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 05 23:10:38 dist-test-slave-ss4n krb5kdc[31588](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971438, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:10:38.220525 25971 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 05 23:10:38 dist-test-slave-ss4n krb5kdc[31588](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971438, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.92.193@KRBTEST.COM
I20260805 23:10:38.231091 32114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:38.221380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36032 (local address 127.25.92.193:46425)
0805 23:10:38.221560 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:38.221565 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:38.221740 (+ 175us) server_negotiation.cc:407] Connection header received
0805 23:10:38.221887 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:38.221890 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:38.221939 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:38.222041 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:38.222626 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:38.223423 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:38.224256 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:38.224649 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:38.224765 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:38.227560 (+ 2795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:38.227570 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:38.227585 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:38.227613 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:38.229186 (+ 1573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:38.229713 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:38.229719 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:38.229720 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:38.229801 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:38.230098 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:38.230102 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:38.230104 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:38.230360 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:38.230586 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:38.230774 (+ 188us) server_negotiation.cc:299] Negotiation successful
0805 23:10:38.230946 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
W20260805 23:10:38.235216 32114 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:10:38.234432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36042 (local address 127.25.92.193:46425)
0805 23:10:38.234543 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:38.234546 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:38.234559 (+ 13us) server_negotiation.cc:407] Connection header received
0805 23:10:38.234592 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:38.234594 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:38.234636 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:38.234715 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:38.235103 (+ 388us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36042: BlockingRecv error: recv got EOF from 127.0.0.1:36042 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260805 23:10:38.235505 25971 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971406251336-25971-0/krb5kdc/krb5cc' of type FILE
Aug 05 23:10:38 dist-test-slave-ss4n krb5kdc[31588](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971438, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:10:38.236552 25971 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 05 23:10:38 dist-test-slave-ss4n krb5kdc[31588](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971438, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:10:38.237457 25971 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 05 23:10:38 dist-test-slave-ss4n krb5kdc[31588](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971438, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.92.193@KRBTEST.COM
I20260805 23:10:38.246421 32114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:38.238298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36052 (local address 127.25.92.193:46425)
0805 23:10:38.238405 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:38.238410 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:38.238546 (+ 136us) server_negotiation.cc:407] Connection header received
0805 23:10:38.238701 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:38.238703 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:38.238738 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:38.238823 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:38.239320 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:38.239850 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:38.240643 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:38.240880 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:38.240981 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:38.243229 (+ 2248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:38.243232 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:38.243234 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:38.243256 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:38.244807 (+ 1551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:38.245235 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:38.245238 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:38.245239 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:38.245315 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:38.245603 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:38.245606 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:38.245607 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:38.245760 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:38.245894 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:38.246108 (+ 214us) server_negotiation.cc:299] Negotiation successful
0805 23:10:38.246232 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
W20260805 23:10:38.248970 32114 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:10:38.248105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36060 (local address 127.25.92.193:46425)
0805 23:10:38.248248 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:38.248252 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:38.248278 (+ 26us) server_negotiation.cc:407] Connection header received
0805 23:10:38.248381 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:38.248386 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:38.248435 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:38.248517 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:38.248862 (+ 345us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36060: BlockingRecv error: recv got EOF from 127.0.0.1:36060 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
W20260805 23:10:38.249069 25971 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971406251336-25971-0/krb5kdc/krb5cc' of type FILE
Aug 05 23:10:38 dist-test-slave-ss4n krb5kdc[31588](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971438, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:10:38.250111 25971 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 05 23:10:38 dist-test-slave-ss4n krb5kdc[31588](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971438, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:10:38.250974 25971 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 05 23:10:38 dist-test-slave-ss4n krb5kdc[31588](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971438, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.92.193@KRBTEST.COM
I20260805 23:10:38.259253 32114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:38.251764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36068 (local address 127.25.92.193:46425)
0805 23:10:38.251867 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:38.251870 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:38.252047 (+ 177us) server_negotiation.cc:407] Connection header received
0805 23:10:38.252139 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:38.252141 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:38.252181 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:38.252242 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:38.252689 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:38.253171 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:38.253998 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:38.254216 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:38.254294 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:38.256389 (+ 2095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:38.256392 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:38.256394 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:38.256415 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:38.257719 (+ 1304us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:38.258093 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:38.258096 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:38.258097 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:38.258177 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:38.258504 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:38.258506 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:38.258507 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:38.258658 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:38.258752 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:38.258952 (+ 200us) server_negotiation.cc:299] Negotiation successful
0805 23:10:38.259040 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260805 23:10:38.260329 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 31697
I20260805 23:10:38.266512 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 31836
I20260805 23:10:38.273458 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 31975
I20260805 23:10:38.278950 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 31601
2026-08-05T23:10:38Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2304 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 23:10:38 dist-test-slave-ss4n krb5kdc[32127](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 23:10:38 dist-test-slave-ss4n krb5kdc[32127](info): set up 2 sockets
Aug 05 23:10:38 dist-test-slave-ss4n krb5kdc[32127](info): commencing operation
krb5kdc: starting...
W20260805 23:10:39.430383 25971 mini_kdc.cc:121] Time spent starting KDC: real 1.128s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/krb5kdc/test-user.keytab.
Aug 05 23:10:39 dist-test-slave-ss4n krb5kdc[32127](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971439, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T23:10:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:10:39Z Disabled control of system clock
WARNING: no policy specified for oryx/127.25.92.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.25.92.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.25.92.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/krb5kdc/oryx_127.25.92.254.keytab.
Entry for principal oryx/127.25.92.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/krb5kdc/oryx_127.25.92.254.keytab.
I20260805 23:10:39.566118 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.254:39541
--webserver_interface=127.25.92.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/krb5kdc/oryx_127.25.92.254.keytab
--principal=oryx/127.25.92.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:44103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.92.254:39541
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:10:39.683177 32140 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:10:39.683485 32140 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:10:39.683529 32140 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:10:39.687642 32140 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:10:39.687712 32140 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:10:39.687732 32140 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:10:39.687750 32140 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:10:39.687765 32140 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:10:39.693446 32140 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:44103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.92.254:39541
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/krb5kdc/oryx_127.25.92.254.keytab
--principal=oryx/127.25.92.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.254:39541
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.92.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32140
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:10:39.694645 32140 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:10:39.695672 32140 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:10:39.705322 32146 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:39.705322 32148 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:10:39.705761 32140 server_base.cc:1034] running on GCE node
W20260805 23:10:39.705361 32145 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:10:39.706354 32140 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:10:39.707638 32140 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:10:39.708827 32140 hybrid_clock.cc:648] HybridClock initialized: now 1785971439708805 us; error 36 us; skew 500 ppm
Aug 05 23:10:39 dist-test-slave-ss4n krb5kdc[32127](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971439, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.92.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:10:39.711941 32140 init.cc:377] Logged in from keytab as oryx/127.25.92.254@KRBTEST.COM (short username oryx)
I20260805 23:10:39.712978 32140 webserver.cc:460] Webserver started at http://127.25.92.254:41609/ using document root <none> and password file <none>
I20260805 23:10:39.713688 32140 fs_manager.cc:359] Metadata directory not provided
I20260805 23:10:39.713763 32140 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:10:39.713949 32140 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:10:39.716010 32140 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/master-0/data/instance:
uuid: "87c2eb89255e45f0a2feae991859aa8e"
format_stamp: "Formatted at 2026-08-05 23:10:39 on dist-test-slave-ss4n"
server_key: "03a391bc38329e012a46f9ec7a93af1e"
server_key_iv: "467bec57e4966d2daf60105646ea7f76"
server_key_version: "encryptionkey@0"
I20260805 23:10:39.716575 32140 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/master-0/wal/instance:
uuid: "87c2eb89255e45f0a2feae991859aa8e"
format_stamp: "Formatted at 2026-08-05 23:10:39 on dist-test-slave-ss4n"
server_key: "03a391bc38329e012a46f9ec7a93af1e"
server_key_iv: "467bec57e4966d2daf60105646ea7f76"
server_key_version: "encryptionkey@0"
I20260805 23:10:39.720713 32140 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260805 23:10:39.723578 32155 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:39.725008 32140 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260805 23:10:39.725132 32140 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/master-0/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/master-0/wal
uuid: "87c2eb89255e45f0a2feae991859aa8e"
format_stamp: "Formatted at 2026-08-05 23:10:39 on dist-test-slave-ss4n"
server_key: "03a391bc38329e012a46f9ec7a93af1e"
server_key_iv: "467bec57e4966d2daf60105646ea7f76"
server_key_version: "encryptionkey@0"
I20260805 23:10:39.725239 32140 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:10:39.737612 32140 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:10:39.741245 32140 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:10:39.741426 32140 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:10:39.749738 32140 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.254:39541
I20260805 23:10:39.750108 32209 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.254:39541 every 8 connection(s)
I20260805 23:10:39.750896 32140 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
I20260805 23:10:39.753602 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 32140
I20260805 23:10:39.753786 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/master-0/wal/instance
I20260805 23:10:39.753986 32210 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:10:39.754145 25971 external_mini_cluster.cc:1428] Setting key 2989bb961218b42b006cd3c650b98534
I20260805 23:10:39.761345 32210 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87c2eb89255e45f0a2feae991859aa8e: Bootstrap starting.
I20260805 23:10:39.763849 32210 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 87c2eb89255e45f0a2feae991859aa8e: Neither blocks nor log segments found. Creating new log.
Aug 05 23:10:39 dist-test-slave-ss4n krb5kdc[32127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971439, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM
I20260805 23:10:39.764647 32210 log.cc:824] T 00000000000000000000000000000000 P 87c2eb89255e45f0a2feae991859aa8e: Log is configured to *not* fsync() on all Append() calls
I20260805 23:10:39.766613 32210 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87c2eb89255e45f0a2feae991859aa8e: No bootstrap required, opened a new log
I20260805 23:10:39.769789 32213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:39.755592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60612 (local address 127.25.92.254:39541)
0805 23:10:39.756049 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:39.756065 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:39.756104 (+ 39us) server_negotiation.cc:407] Connection header received
0805 23:10:39.756768 (+ 664us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:39.756785 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:39.757121 (+ 336us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:39.757440 (+ 319us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:39.758417 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:39.759595 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:39.760835 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:39.761389 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:39.761732 (+ 343us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:39.764643 (+ 2911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:39.764648 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:39.764663 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:39.764694 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:39.766721 (+ 2027us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:39.767230 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:39.767236 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:39.767243 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:39.767368 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:39.767696 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:39.767700 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:39.767702 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:39.768128 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:39.768355 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:39.768658 (+ 303us) server_negotiation.cc:299] Negotiation successful
0805 23:10:39.768969 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":151,"threads_started":1}
I20260805 23:10:39.770314 32210 raft_consensus.cc:348] T 00000000000000000000000000000000 P 87c2eb89255e45f0a2feae991859aa8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87c2eb89255e45f0a2feae991859aa8e" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 39541 } }
I20260805 23:10:39.770570 32210 raft_consensus.cc:374] T 00000000000000000000000000000000 P 87c2eb89255e45f0a2feae991859aa8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:10:39.770651 32210 raft_consensus.cc:729] T 00000000000000000000000000000000 P 87c2eb89255e45f0a2feae991859aa8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87c2eb89255e45f0a2feae991859aa8e, State: Initialized, Role: FOLLOWER
I20260805 23:10:39.771162 32210 consensus_queue.cc:260] T 00000000000000000000000000000000 P 87c2eb89255e45f0a2feae991859aa8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87c2eb89255e45f0a2feae991859aa8e" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 39541 } }
I20260805 23:10:39.771323 32210 raft_consensus.cc:388] T 00000000000000000000000000000000 P 87c2eb89255e45f0a2feae991859aa8e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:10:39.771380 32210 raft_consensus.cc:482] T 00000000000000000000000000000000 P 87c2eb89255e45f0a2feae991859aa8e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:10:39.771477 32210 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 87c2eb89255e45f0a2feae991859aa8e [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:10:39.772714 32210 raft_consensus.cc:504] T 00000000000000000000000000000000 P 87c2eb89255e45f0a2feae991859aa8e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87c2eb89255e45f0a2feae991859aa8e" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 39541 } }
I20260805 23:10:39.773013 32210 leader_election.cc:302] T 00000000000000000000000000000000 P 87c2eb89255e45f0a2feae991859aa8e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 87c2eb89255e45f0a2feae991859aa8e; no voters:
I20260805 23:10:39.773283 32210 leader_election.cc:288] T 00000000000000000000000000000000 P 87c2eb89255e45f0a2feae991859aa8e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 23:10:39.773536 32215 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 87c2eb89255e45f0a2feae991859aa8e [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:10:39.773823 32215 raft_consensus.cc:686] T 00000000000000000000000000000000 P 87c2eb89255e45f0a2feae991859aa8e [term 1 LEADER]: Becoming Leader. State: Replica: 87c2eb89255e45f0a2feae991859aa8e, State: Running, Role: LEADER
I20260805 23:10:39.774168 32215 consensus_queue.cc:237] T 00000000000000000000000000000000 P 87c2eb89255e45f0a2feae991859aa8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87c2eb89255e45f0a2feae991859aa8e" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 39541 } }
I20260805 23:10:39.774451 32210 sys_catalog.cc:564] T 00000000000000000000000000000000 P 87c2eb89255e45f0a2feae991859aa8e [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:10:39.776300 32217 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87c2eb89255e45f0a2feae991859aa8e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 87c2eb89255e45f0a2feae991859aa8e. Latest consensus state: current_term: 1 leader_uuid: "87c2eb89255e45f0a2feae991859aa8e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87c2eb89255e45f0a2feae991859aa8e" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 39541 } } }
I20260805 23:10:39.776464 32217 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87c2eb89255e45f0a2feae991859aa8e [sys.catalog]: This master's current role is: LEADER
I20260805 23:10:39.776733 32216 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87c2eb89255e45f0a2feae991859aa8e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "87c2eb89255e45f0a2feae991859aa8e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87c2eb89255e45f0a2feae991859aa8e" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 39541 } } }
I20260805 23:10:39.776822 32216 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87c2eb89255e45f0a2feae991859aa8e [sys.catalog]: This master's current role is: LEADER
I20260805 23:10:39.776870 32229 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:10:39.780236 32229 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:10:39.786118 32229 catalog_manager.cc:1347] Generated new cluster ID: cfa50e6d78c6414ca13b867ff0c3f044
I20260805 23:10:39.786185 32229 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:10:39.796964 32229 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:10:39.798272 32229 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:10:39.807561 32229 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 87c2eb89255e45f0a2feae991859aa8e: Generated new TSK 0
I20260805 23:10:39.808445 32229 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.25.92.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.25.92.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.25.92.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/krb5kdc/oryx_127.25.92.193.keytab.
Entry for principal oryx/127.25.92.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/krb5kdc/oryx_127.25.92.193.keytab.
I20260805 23:10:39.850293 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.92.193:0
--local_ip_for_outbound_sockets=127.25.92.193
--webserver_interface=127.25.92.193
--webserver_port=0
--tserver_master_addrs=127.25.92.254:39541
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/krb5kdc/oryx_127.25.92.193.keytab
--principal=oryx/127.25.92.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:44103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:10:39.971069 32237 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:10:39.971355 32237 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:10:39.971400 32237 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:10:39.975848 32237 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:10:39.975927 32237 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:10:39.976022 32237 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.92.193
I20260805 23:10:39.981747 32237 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:44103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/krb5kdc/oryx_127.25.92.193.keytab
--principal=oryx/127.25.92.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.92.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.92.193
--webserver_port=0
--tserver_master_addrs=127.25.92.254:39541
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32237
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.92.193
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:10:39.982955 32237 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:10:39.984010 32237 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:10:39.993935 32243 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:39.993935 32242 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:10:39.994174 32237 server_base.cc:1034] running on GCE node
W20260805 23:10:39.993985 32245 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:10:39.994733 32237 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:10:39.995373 32237 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:10:39.996641 32237 hybrid_clock.cc:648] HybridClock initialized: now 1785971439996616 us; error 37 us; skew 500 ppm
Aug 05 23:10:39 dist-test-slave-ss4n krb5kdc[32127](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971439, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.92.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:10:40.003301 32237 init.cc:377] Logged in from keytab as oryx/127.25.92.193@KRBTEST.COM (short username oryx)
I20260805 23:10:40.004235 32237 webserver.cc:460] Webserver started at http://127.25.92.193:41671/ using document root <none> and password file <none>
I20260805 23:10:40.004783 32237 fs_manager.cc:359] Metadata directory not provided
I20260805 23:10:40.004844 32237 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:10:40.005035 32237 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:10:40.007203 32237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/ts-0/data/instance:
uuid: "64c076108dc34bc68cacff2bd1d9282f"
format_stamp: "Formatted at 2026-08-05 23:10:39 on dist-test-slave-ss4n"
server_key: "8dcd60f6e0e26e78d0446e031625279c"
server_key_iv: "b6bc2c49fca0000cabdd416f8e39766e"
server_key_version: "encryptionkey@0"
I20260805 23:10:40.007745 32237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/ts-0/wal/instance:
uuid: "64c076108dc34bc68cacff2bd1d9282f"
format_stamp: "Formatted at 2026-08-05 23:10:39 on dist-test-slave-ss4n"
server_key: "8dcd60f6e0e26e78d0446e031625279c"
server_key_iv: "b6bc2c49fca0000cabdd416f8e39766e"
server_key_version: "encryptionkey@0"
I20260805 23:10:40.011976 32237 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260805 23:10:40.014950 32252 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:40.016381 32237 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260805 23:10:40.016559 32237 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/ts-0/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/ts-0/wal
uuid: "64c076108dc34bc68cacff2bd1d9282f"
format_stamp: "Formatted at 2026-08-05 23:10:39 on dist-test-slave-ss4n"
server_key: "8dcd60f6e0e26e78d0446e031625279c"
server_key_iv: "b6bc2c49fca0000cabdd416f8e39766e"
server_key_version: "encryptionkey@0"
I20260805 23:10:40.016719 32237 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:10:40.032478 32237 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:10:40.035727 32237 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:10:40.035943 32237 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:10:40.036547 32237 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:10:40.037382 32237 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:10:40.037453 32237 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:40.037555 32237 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:10:40.037607 32237 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:40.046545 32237 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.193:44453
I20260805 23:10:40.046624 32368 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.193:44453 every 8 connection(s)
I20260805 23:10:40.047495 32237 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/ts-0/data/info.pb
I20260805 23:10:40.047732 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 32237
I20260805 23:10:40.047883 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/ts-0/wal/instance
I20260805 23:10:40.048269 25971 external_mini_cluster.cc:1428] Setting key a7e74adccac84452fa6e44293c0f0db6
WARNING: no policy specified for oryx/127.25.92.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.25.92.194@KRBTEST.COM" created.
Aug 05 23:10:40 dist-test-slave-ss4n krb5kdc[32127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971439, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.92.193@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM
I20260805 23:10:40.069759 32213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:40.049619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.193:44977 (local address 127.25.92.254:39541)
0805 23:10:40.049790 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:40.049796 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:40.050918 (+ 1122us) server_negotiation.cc:407] Connection header received
0805 23:10:40.053045 (+ 2127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:40.053048 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:40.053097 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:40.053181 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:40.055049 (+ 1868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:40.055607 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:40.057820 (+ 2213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:40.058107 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:40.058223 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:40.064600 (+ 6377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:40.064608 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:40.064616 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:40.064661 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:40.066671 (+ 2010us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:40.067541 (+ 870us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:40.067544 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:40.067546 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:40.067656 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:40.068217 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:40.068224 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:40.068227 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:40.068472 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:40.068585 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:40.069338 (+ 753us) server_negotiation.cc:299] Negotiation successful
0805 23:10:40.069552 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260805 23:10:40.070678 32371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:40.050189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:39541 (local address 127.25.92.193:44977)
0805 23:10:40.050738 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:40.050797 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:40.052766 (+ 1969us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:40.053412 (+ 646us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:40.053426 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:40.054054 (+ 628us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:40.054855 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:40.054884 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:40.055765 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:40.055772 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:40.057600 (+ 1828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:40.057615 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:40.058303 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:40.058307 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:40.058581 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:40.059296 (+ 715us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:10:40.059327 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:40.064329 (+ 5002us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:10:40.066834 (+ 2505us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:40.066845 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:40.066871 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:40.067359 (+ 488us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:40.067773 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:40.067780 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:40.067786 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:40.068028 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:40.068608 (+ 580us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:10:40.068617 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:10:40.069015 (+ 398us) client_negotiation.cc:769] Sending connection context
0805 23:10:40.069259 (+ 244us) client_negotiation.cc:240] Negotiation successful
0805 23:10:40.069703 (+ 444us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":150,"threads_started":1}
I20260805 23:10:40.072172 32369 heartbeater.cc:347] Connected to a master server at 127.25.92.254:39541
I20260805 23:10:40.072453 32369 heartbeater.cc:464] Registering TS with master...
I20260805 23:10:40.073086 32369 heartbeater.cc:511] Master 127.25.92.254:39541 requested a full tablet report, sending...
I20260805 23:10:40.075007 32174 ts_manager.cc:194] Registered new tserver with Master: 64c076108dc34bc68cacff2bd1d9282f (127.25.92.193:44453)
I20260805 23:10:40.076464 32174 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.25.92.193@KRBTEST.COM'} at 127.25.92.193:44977
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.25.92.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/krb5kdc/oryx_127.25.92.194.keytab.
Entry for principal oryx/127.25.92.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/krb5kdc/oryx_127.25.92.194.keytab.
I20260805 23:10:40.084941 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.92.194:0
--local_ip_for_outbound_sockets=127.25.92.194
--webserver_interface=127.25.92.194
--webserver_port=0
--tserver_master_addrs=127.25.92.254:39541
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/krb5kdc/oryx_127.25.92.194.keytab
--principal=oryx/127.25.92.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:44103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:10:40.198300 32374 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:10:40.198576 32374 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:10:40.198652 32374 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:10:40.202800 32374 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:10:40.202863 32374 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:10:40.202956 32374 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.92.194
I20260805 23:10:40.208454 32374 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:44103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/krb5kdc/oryx_127.25.92.194.keytab
--principal=oryx/127.25.92.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.92.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.92.194
--webserver_port=0
--tserver_master_addrs=127.25.92.254:39541
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32374
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.92.194
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:10:40.209748 32374 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:10:40.210772 32374 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:10:40.220500 32380 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:40.220534 32379 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:40.220666 32382 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:10:40.221047 32374 server_base.cc:1034] running on GCE node
I20260805 23:10:40.221525 32374 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:10:40.222168 32374 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:10:40.223352 32374 hybrid_clock.cc:648] HybridClock initialized: now 1785971440223300 us; error 61 us; skew 500 ppm
Aug 05 23:10:40 dist-test-slave-ss4n krb5kdc[32127](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971440, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.92.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:10:40.226881 32374 init.cc:377] Logged in from keytab as oryx/127.25.92.194@KRBTEST.COM (short username oryx)
I20260805 23:10:40.227790 32374 webserver.cc:460] Webserver started at http://127.25.92.194:44929/ using document root <none> and password file <none>
I20260805 23:10:40.228413 32374 fs_manager.cc:359] Metadata directory not provided
I20260805 23:10:40.228561 32374 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:10:40.228828 32374 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:10:40.230964 32374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/ts-1/data/instance:
uuid: "fc9234f3fa864a598207c335d409c16c"
format_stamp: "Formatted at 2026-08-05 23:10:40 on dist-test-slave-ss4n"
server_key: "41c6ab6f38b82dfd39dc9c3693c764fb"
server_key_iv: "9c4ee8c76c788b06210409c75d443e95"
server_key_version: "encryptionkey@0"
I20260805 23:10:40.231532 32374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/ts-1/wal/instance:
uuid: "fc9234f3fa864a598207c335d409c16c"
format_stamp: "Formatted at 2026-08-05 23:10:40 on dist-test-slave-ss4n"
server_key: "41c6ab6f38b82dfd39dc9c3693c764fb"
server_key_iv: "9c4ee8c76c788b06210409c75d443e95"
server_key_version: "encryptionkey@0"
I20260805 23:10:40.235422 32374 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260805 23:10:40.237958 32389 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:40.239298 32374 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:10:40.239467 32374 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/ts-1/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/ts-1/wal
uuid: "fc9234f3fa864a598207c335d409c16c"
format_stamp: "Formatted at 2026-08-05 23:10:40 on dist-test-slave-ss4n"
server_key: "41c6ab6f38b82dfd39dc9c3693c764fb"
server_key_iv: "9c4ee8c76c788b06210409c75d443e95"
server_key_version: "encryptionkey@0"
I20260805 23:10:40.239602 32374 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:10:40.264410 32374 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:10:40.268074 32374 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:10:40.268299 32374 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:10:40.268934 32374 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:10:40.269871 32374 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:10:40.269943 32374 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:40.270026 32374 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:10:40.270078 32374 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:40.280166 32374 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.194:36939
I20260805 23:10:40.280213 32504 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.194:36939 every 8 connection(s)
I20260805 23:10:40.281183 32374 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/ts-1/data/info.pb
I20260805 23:10:40.282507 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 32374
I20260805 23:10:40.282644 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/ts-1/wal/instance
I20260805 23:10:40.282902 25971 external_mini_cluster.cc:1428] Setting key 6bec8145129207d713f6b61cb9ed4ed1
WARNING: no policy specified for oryx/127.25.92.195@KRBTEST.COM; defaulting to no policy
Aug 05 23:10:40 dist-test-slave-ss4n krb5kdc[32127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971440, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.92.194@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.25.92.195@KRBTEST.COM" created.
I20260805 23:10:40.297374 32213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:40.283389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.194:51775 (local address 127.25.92.254:39541)
0805 23:10:40.283535 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:40.283539 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:40.285079 (+ 1540us) server_negotiation.cc:407] Connection header received
0805 23:10:40.286079 (+ 1000us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:40.286083 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:40.286147 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:40.286262 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:40.287561 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:40.288202 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:40.289461 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:40.289791 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:40.289920 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:40.292838 (+ 2918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:40.292843 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:40.292846 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:40.292887 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:40.294693 (+ 1806us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:40.295407 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:40.295413 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:40.295415 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:40.295544 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:40.296004 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:40.296010 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:40.296012 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:40.296247 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:40.296382 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:40.296987 (+ 605us) server_negotiation.cc:299] Negotiation successful
0805 23:10:40.297155 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260805 23:10:40.298333 32507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:40.283851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:39541 (local address 127.25.92.194:51775)
0805 23:10:40.284904 (+ 1053us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:40.284961 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:40.285843 (+ 882us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:40.286450 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:40.286458 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:40.286884 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:40.287394 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:40.287409 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:40.288348 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:40.288357 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:40.289309 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:40.289319 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:40.289965 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:40.289970 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:40.290165 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:40.290871 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:10:40.290890 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:40.292629 (+ 1739us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:10:40.294887 (+ 2258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:40.294895 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:40.294915 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:40.295274 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:40.295676 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:40.295680 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:40.295683 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:40.295879 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:40.296379 (+ 500us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:10:40.296388 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:10:40.296688 (+ 300us) client_negotiation.cc:769] Sending connection context
0805 23:10:40.296923 (+ 235us) client_negotiation.cc:240] Negotiation successful
0805 23:10:40.297321 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":835,"thread_start_us":163,"threads_started":1}
I20260805 23:10:40.299764 32505 heartbeater.cc:347] Connected to a master server at 127.25.92.254:39541
I20260805 23:10:40.300074 32505 heartbeater.cc:464] Registering TS with master...
I20260805 23:10:40.300781 32505 heartbeater.cc:511] Master 127.25.92.254:39541 requested a full tablet report, sending...
I20260805 23:10:40.302248 32174 ts_manager.cc:194] Registered new tserver with Master: fc9234f3fa864a598207c335d409c16c (127.25.92.194:36939)
I20260805 23:10:40.303078 32174 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.25.92.194@KRBTEST.COM'} at 127.25.92.194:51775
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.25.92.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/krb5kdc/oryx_127.25.92.195.keytab.
Entry for principal oryx/127.25.92.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/krb5kdc/oryx_127.25.92.195.keytab.
I20260805 23:10:40.314066 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.92.195:0
--local_ip_for_outbound_sockets=127.25.92.195
--webserver_interface=127.25.92.195
--webserver_port=0
--tserver_master_addrs=127.25.92.254:39541
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/krb5kdc/oryx_127.25.92.195.keytab
--principal=oryx/127.25.92.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:44103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:10:40.428334 32510 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:10:40.428632 32510 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:10:40.428679 32510 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:10:40.433116 32510 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:10:40.433197 32510 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:10:40.433260 32510 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.92.195
I20260805 23:10:40.438755 32510 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:44103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/krb5kdc/oryx_127.25.92.195.keytab
--principal=oryx/127.25.92.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.92.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.92.195
--webserver_port=0
--tserver_master_addrs=127.25.92.254:39541
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32510
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.92.195
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:10:40.439924 32510 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:10:40.441073 32510 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:10:40.451449 32518 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:40.451489 32515 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:40.451614 32516 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:10:40.451789 32510 server_base.cc:1034] running on GCE node
I20260805 23:10:40.452368 32510 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:10:40.453111 32510 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:10:40.454331 32510 hybrid_clock.cc:648] HybridClock initialized: now 1785971440454297 us; error 46 us; skew 500 ppm
Aug 05 23:10:40 dist-test-slave-ss4n krb5kdc[32127](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971440, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.92.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:10:40.457382 32510 init.cc:377] Logged in from keytab as oryx/127.25.92.195@KRBTEST.COM (short username oryx)
I20260805 23:10:40.458422 32510 webserver.cc:460] Webserver started at http://127.25.92.195:43867/ using document root <none> and password file <none>
I20260805 23:10:40.459035 32510 fs_manager.cc:359] Metadata directory not provided
I20260805 23:10:40.459141 32510 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:10:40.459385 32510 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:10:40.461660 32510 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/ts-2/data/instance:
uuid: "e6569ceea53644f6b26a1e4c960fe340"
format_stamp: "Formatted at 2026-08-05 23:10:40 on dist-test-slave-ss4n"
server_key: "c4228b7a91ec3c18f328a8988a47d8b3"
server_key_iv: "5b69eae7ec17440aeed7b1de5c34bce9"
server_key_version: "encryptionkey@0"
I20260805 23:10:40.462283 32510 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/ts-2/wal/instance:
uuid: "e6569ceea53644f6b26a1e4c960fe340"
format_stamp: "Formatted at 2026-08-05 23:10:40 on dist-test-slave-ss4n"
server_key: "c4228b7a91ec3c18f328a8988a47d8b3"
server_key_iv: "5b69eae7ec17440aeed7b1de5c34bce9"
server_key_version: "encryptionkey@0"
I20260805 23:10:40.466091 32510 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260805 23:10:40.468698 32525 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:40.470124 32510 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:10:40.470270 32510 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/ts-2/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/ts-2/wal
uuid: "e6569ceea53644f6b26a1e4c960fe340"
format_stamp: "Formatted at 2026-08-05 23:10:40 on dist-test-slave-ss4n"
server_key: "c4228b7a91ec3c18f328a8988a47d8b3"
server_key_iv: "5b69eae7ec17440aeed7b1de5c34bce9"
server_key_version: "encryptionkey@0"
I20260805 23:10:40.470394 32510 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:10:40.482777 32510 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:10:40.486403 32510 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:10:40.486629 32510 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:10:40.487259 32510 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:10:40.488133 32510 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:10:40.488204 32510 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:40.488277 32510 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:10:40.488322 32510 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:40.498219 32510 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.195:41337
I20260805 23:10:40.498238 32642 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.195:41337 every 8 connection(s)
I20260805 23:10:40.499158 32510 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/ts-2/data/info.pb
I20260805 23:10:40.501807 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 32510
I20260805 23:10:40.501931 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/ts-2/wal/instance
I20260805 23:10:40.502254 25971 external_mini_cluster.cc:1428] Setting key ee08a150bbc61632d90282b2a06df299
Aug 05 23:10:40 dist-test-slave-ss4n krb5kdc[32127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971440, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.92.195@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM
I20260805 23:10:40.513723 32213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:40.501013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.195:60525 (local address 127.25.92.254:39541)
0805 23:10:40.501189 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:40.501193 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:40.502342 (+ 1149us) server_negotiation.cc:407] Connection header received
0805 23:10:40.504008 (+ 1666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:40.504011 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:40.504068 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:40.504174 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:40.505413 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:40.506011 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:40.507190 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:40.507411 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:40.507521 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:40.510108 (+ 2587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:40.510112 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:40.510114 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:40.510138 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:40.511640 (+ 1502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:40.512145 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:40.512150 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:40.512152 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:40.512234 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:40.512570 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:40.512575 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:40.512577 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:40.512781 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:40.512893 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:40.513385 (+ 492us) server_negotiation.cc:299] Negotiation successful
0805 23:10:40.513537 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260805 23:10:40.514559 32645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:40.501412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:39541 (local address 127.25.92.195:60525)
0805 23:10:40.502187 (+ 775us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:40.502232 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:40.503792 (+ 1560us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:40.504364 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:40.504376 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:40.504787 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:40.505273 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:40.505286 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:40.506151 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:40.506154 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:40.507049 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:40.507059 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:40.507561 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:40.507565 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:40.507743 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:40.508350 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:10:40.508367 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:40.509916 (+ 1549us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:10:40.511759 (+ 1843us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:40.511765 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:40.511780 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:40.512035 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:40.512327 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:40.512332 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:40.512334 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:40.512472 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:40.512889 (+ 417us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:10:40.512895 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:10:40.513156 (+ 261us) client_negotiation.cc:769] Sending connection context
0805 23:10:40.513345 (+ 189us) client_negotiation.cc:240] Negotiation successful
0805 23:10:40.513690 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":192,"threads_started":1}
I20260805 23:10:40.515964 32643 heartbeater.cc:347] Connected to a master server at 127.25.92.254:39541
I20260805 23:10:40.516227 32643 heartbeater.cc:464] Registering TS with master...
I20260805 23:10:40.516798 32643 heartbeater.cc:511] Master 127.25.92.254:39541 requested a full tablet report, sending...
I20260805 23:10:40.517954 32174 ts_manager.cc:194] Registered new tserver with Master: e6569ceea53644f6b26a1e4c960fe340 (127.25.92.195:41337)
I20260805 23:10:40.518532 32174 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.25.92.195@KRBTEST.COM'} at 127.25.92.195:60525
I20260805 23:10:40.528613 25971 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.25.92.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.25.92.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.25.92.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/krb5kdc/oryx_127.25.92.253.keytab.
Entry for principal oryx/127.25.92.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/krb5kdc/oryx_127.25.92.253.keytab.
I20260805 23:10:40.561610 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.253:38299
--webserver_interface=127.25.92.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/krb5kdc/oryx_127.25.92.253.keytab
--principal=oryx/127.25.92.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:44103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.92.254:39541,127.25.92.253:38299
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:10:40.679602 32648 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:10:40.679888 32648 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:10:40.679934 32648 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:10:40.684119 32648 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:10:40.684192 32648 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:10:40.684212 32648 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:10:40.684230 32648 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:10:40.684247 32648 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:10:40.689710 32648 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:44103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.25.92.254:39541,127.25.92.253:38299
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/krb5kdc/oryx_127.25.92.253.keytab
--principal=oryx/127.25.92.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.253:38299
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.25.92.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32648
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
W20260805 23:10:40.689924 32648 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.25.92.254:39541,127.25.92.253:38299'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260805 23:10:40.690778 32648 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:10:40.691782 32648 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:10:40.700515 32653 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:40.700616 32656 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:40.700548 32654 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:10:40.700944 32648 server_base.cc:1034] running on GCE node
I20260805 23:10:40.701611 32648 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:10:40.702869 32648 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:10:40.704087 32648 hybrid_clock.cc:648] HybridClock initialized: now 1785971440704059 us; error 53 us; skew 500 ppm
Aug 05 23:10:40 dist-test-slave-ss4n krb5kdc[32127](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971440, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.92.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:10:40.707234 32648 init.cc:377] Logged in from keytab as oryx/127.25.92.253@KRBTEST.COM (short username oryx)
I20260805 23:10:40.708204 32648 webserver.cc:460] Webserver started at http://127.25.92.253:42655/ using document root <none> and password file <none>
I20260805 23:10:40.708782 32648 fs_manager.cc:359] Metadata directory not provided
I20260805 23:10:40.708870 32648 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:10:40.709117 32648 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:10:40.711252 32648 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/master-1/data/instance:
uuid: "d72d1f2dd8074bbbaf2fc4761feb7080"
format_stamp: "Formatted at 2026-08-05 23:10:40 on dist-test-slave-ss4n"
server_key: "25e6ad65c9be854fa29b540a40c04eab"
server_key_iv: "1dc1beab376c057317a44f3d33c439c2"
server_key_version: "encryptionkey@0"
I20260805 23:10:40.711824 32648 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/master-1/wal/instance:
uuid: "d72d1f2dd8074bbbaf2fc4761feb7080"
format_stamp: "Formatted at 2026-08-05 23:10:40 on dist-test-slave-ss4n"
server_key: "25e6ad65c9be854fa29b540a40c04eab"
server_key_iv: "1dc1beab376c057317a44f3d33c439c2"
server_key_version: "encryptionkey@0"
I20260805 23:10:40.715981 32648 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260805 23:10:40.718745 32663 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:40.720320 32648 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:10:40.720508 32648 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/master-1/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/master-1/wal
uuid: "d72d1f2dd8074bbbaf2fc4761feb7080"
format_stamp: "Formatted at 2026-08-05 23:10:40 on dist-test-slave-ss4n"
server_key: "25e6ad65c9be854fa29b540a40c04eab"
server_key_iv: "1dc1beab376c057317a44f3d33c439c2"
server_key_version: "encryptionkey@0"
I20260805 23:10:40.720649 32648 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:10:40.739118 32648 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:10:40.742798 32648 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:10:40.743028 32648 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:10:40.751646 32648 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.253:38299
I20260805 23:10:40.752022 32717 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.253:38299 every 8 connection(s)
I20260805 23:10:40.752823 32648 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/master-1/data/info.pb
I20260805 23:10:40.755792 32718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:10:40.759611 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 32648
I20260805 23:10:40.759984 32718 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.92.254" port: 39541 } has no permanent_uuid. Determining permanent_uuid...
Aug 05 23:10:40 dist-test-slave-ss4n krb5kdc[32127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971440, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.92.253@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM
Aug 05 23:10:40 dist-test-slave-ss4n krb5kdc[32127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971439, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.25.92.253@KRBTEST.COM
I20260805 23:10:40.775087 32213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:40.762267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60614 (local address 127.25.92.254:39541)
0805 23:10:40.762400 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:40.762405 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:40.762825 (+ 420us) server_negotiation.cc:407] Connection header received
0805 23:10:40.763454 (+ 629us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:40.763458 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:40.763508 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:40.763665 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:40.764680 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:40.765259 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:40.766579 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:40.766855 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:40.766940 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:40.770178 (+ 3238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:40.770182 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:40.770185 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:40.770218 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:40.771915 (+ 1697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:40.772662 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:40.772666 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:40.772668 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:40.772764 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:40.773167 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:40.773172 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:40.773174 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:40.773433 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:40.773609 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:40.774543 (+ 934us) server_negotiation.cc:299] Negotiation successful
0805 23:10:40.774880 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260805 23:10:40.775906 32722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:40.762186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:39541 (local address 127.0.0.1:60614)
0805 23:10:40.762670 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:40.762727 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:40.763175 (+ 448us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:40.763726 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:40.763731 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:40.764199 (+ 468us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:40.764492 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:40.764514 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:40.765403 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:40.765407 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:40.766430 (+ 1023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:40.766440 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:40.767010 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:40.767015 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:40.767226 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:40.768158 (+ 932us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:10:40.768176 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:40.769987 (+ 1811us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:10:40.772108 (+ 2121us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:40.772118 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:40.772131 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:40.772542 (+ 411us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:40.772867 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:40.772873 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:40.772875 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:40.773060 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:40.774075 (+ 1015us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:10:40.774082 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:10:40.774255 (+ 173us) client_negotiation.cc:769] Sending connection context
0805 23:10:40.774489 (+ 234us) client_negotiation.cc:240] Negotiation successful
0805 23:10:40.774889 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"mutex_wait_us":40,"thread_start_us":139,"threads_started":1}
I20260805 23:10:40.775921 32721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:40.761390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33806 (local address 127.25.92.253:38299)
0805 23:10:40.761917 (+ 527us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:40.761935 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:40.761982 (+ 47us) server_negotiation.cc:407] Connection header received
0805 23:10:40.763175 (+ 1193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:40.763199 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:40.763464 (+ 265us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:40.763664 (+ 200us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:40.764540 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:40.765709 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:40.766659 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:40.767159 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:40.767267 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:40.770538 (+ 3271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:40.770546 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:40.770563 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:40.770600 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:40.772563 (+ 1963us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:40.773059 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:40.773066 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:40.773068 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:40.773152 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:40.773467 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:40.773470 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:40.773472 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:40.774004 (+ 532us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:40.774236 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:40.774487 (+ 251us) server_negotiation.cc:299] Negotiation successful
0805 23:10:40.774998 (+ 511us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":132,"server-negotiator.queue_time_us":374,"thread_start_us":151,"threads_started":1}
I20260805 23:10:40.777881 32718 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.92.253" port: 38299 } has no permanent_uuid. Determining permanent_uuid...
Aug 05 23:10:40 dist-test-slave-ss4n krb5kdc[32127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971440, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.92.253@KRBTEST.COM for oryx/127.25.92.253@KRBTEST.COM
I20260805 23:10:40.786490 32721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:40.778663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33812 (local address 127.25.92.253:38299)
0805 23:10:40.778762 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:40.778766 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:40.778780 (+ 14us) server_negotiation.cc:407] Connection header received
0805 23:10:40.778867 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:40.778870 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:40.778906 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:40.778996 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:40.779407 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:40.779958 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:40.780761 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:40.781024 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:40.781148 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:40.783351 (+ 2203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:40.783355 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:40.783357 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:40.783388 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:40.784728 (+ 1340us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:40.785171 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:40.785178 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:40.785180 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:40.785260 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:40.785594 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:40.785601 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:40.785604 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:40.785791 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:40.785902 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:40.786170 (+ 268us) server_negotiation.cc:299] Negotiation successful
0805 23:10:40.786336 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":95,"server-negotiator.queue_time_us":41}
I20260805 23:10:40.786490 32722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:40.778532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.253:38299 (local address 127.0.0.1:33812)
0805 23:10:40.778656 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:40.778669 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:40.778773 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:40.779006 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:40.779009 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:40.779202 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:40.779293 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:40.779298 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:40.780075 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:40.780079 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:40.780641 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:40.780649 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:40.781144 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:40.781148 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:40.781279 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:40.781862 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:10:40.781874 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:40.783211 (+ 1337us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:10:40.784856 (+ 1645us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:40.784859 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:40.784862 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:40.785060 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:40.785365 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:40.785367 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:40.785369 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:40.785452 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:40.785902 (+ 450us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:10:40.785905 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:10:40.785998 (+ 93us) client_negotiation.cc:769] Sending connection context
0805 23:10:40.786121 (+ 123us) client_negotiation.cc:240] Negotiation successful
0805 23:10:40.786280 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43,"mutex_wait_us":4}
I20260805 23:10:40.790374 32718 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d72d1f2dd8074bbbaf2fc4761feb7080: Bootstrap starting.
I20260805 23:10:40.792759 32718 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d72d1f2dd8074bbbaf2fc4761feb7080: Neither blocks nor log segments found. Creating new log.
I20260805 23:10:40.793462 32718 log.cc:824] T 00000000000000000000000000000000 P d72d1f2dd8074bbbaf2fc4761feb7080: Log is configured to *not* fsync() on all Append() calls
I20260805 23:10:40.795636 32718 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d72d1f2dd8074bbbaf2fc4761feb7080: No bootstrap required, opened a new log
I20260805 23:10:40.798925 32718 raft_consensus.cc:348] T 00000000000000000000000000000000 P d72d1f2dd8074bbbaf2fc4761feb7080 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87c2eb89255e45f0a2feae991859aa8e" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 39541 } } peers { permanent_uuid: "d72d1f2dd8074bbbaf2fc4761feb7080" member_type: VOTER last_known_addr { host: "127.25.92.253" port: 38299 } }
I20260805 23:10:40.799199 32718 raft_consensus.cc:374] T 00000000000000000000000000000000 P d72d1f2dd8074bbbaf2fc4761feb7080 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:10:40.799286 32718 raft_consensus.cc:729] T 00000000000000000000000000000000 P d72d1f2dd8074bbbaf2fc4761feb7080 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d72d1f2dd8074bbbaf2fc4761feb7080, State: Initialized, Role: FOLLOWER
I20260805 23:10:40.799796 32718 consensus_queue.cc:260] T 00000000000000000000000000000000 P d72d1f2dd8074bbbaf2fc4761feb7080 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87c2eb89255e45f0a2feae991859aa8e" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 39541 } } peers { permanent_uuid: "d72d1f2dd8074bbbaf2fc4761feb7080" member_type: VOTER last_known_addr { host: "127.25.92.253" port: 38299 } }
I20260805 23:10:40.800266 32724 sys_catalog.cc:455] T 00000000000000000000000000000000 P d72d1f2dd8074bbbaf2fc4761feb7080 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87c2eb89255e45f0a2feae991859aa8e" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 39541 } } peers { permanent_uuid: "d72d1f2dd8074bbbaf2fc4761feb7080" member_type: VOTER last_known_addr { host: "127.25.92.253" port: 38299 } } }
I20260805 23:10:40.800420 32724 sys_catalog.cc:458] T 00000000000000000000000000000000 P d72d1f2dd8074bbbaf2fc4761feb7080 [sys.catalog]: This master's current role is: FOLLOWER
I20260805 23:10:40.801038 32718 sys_catalog.cc:564] T 00000000000000000000000000000000 P d72d1f2dd8074bbbaf2fc4761feb7080 [sys.catalog]: configured and running, proceeding with master startup.
W20260805 23:10:40.806556 32735 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d72d1f2dd8074bbbaf2fc4761feb7080: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260805 23:10:40.806664 32735 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260805 23:10:40.809217 32648 master_runner.cc:428] Detected that this master d72d1f2dd8074bbbaf2fc4761feb7080 is joining an existing cluster
I20260805 23:10:40.809286 32648 master_runner.cc:432] Initiating AddMaster RPC to add 127.25.92.253:38299
I20260805 23:10:40.822142 32721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:40.812148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33826 (local address 127.25.92.253:38299)
0805 23:10:40.812271 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:40.812274 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:40.812286 (+ 12us) server_negotiation.cc:407] Connection header received
0805 23:10:40.812439 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:40.812476 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:40.812529 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:40.812620 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:40.813214 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:40.814144 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:40.815543 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:40.815894 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:40.815989 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:40.817598 (+ 1609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:40.817601 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:40.817604 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:40.817634 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:40.819840 (+ 2206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:40.820542 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:40.820546 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:40.820548 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:40.820636 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:40.820953 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:40.820957 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:40.820959 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:40.821216 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:40.821323 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:40.821744 (+ 421us) server_negotiation.cc:299] Negotiation successful
0805 23:10:40.821909 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":92,"server-negotiator.queue_time_us":61}
I20260805 23:10:40.822201 32213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:40.811728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60624 (local address 127.25.92.254:39541)
0805 23:10:40.811868 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:40.811873 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:40.812316 (+ 443us) server_negotiation.cc:407] Connection header received
0805 23:10:40.812505 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:40.812508 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:40.812564 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:40.812651 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:40.813437 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:40.814328 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:40.815453 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:40.815830 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:40.815973 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:40.818200 (+ 2227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:40.818205 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:40.818207 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:40.818242 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:40.820163 (+ 1921us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:40.820687 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:40.820692 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:40.820694 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:40.820799 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:40.821137 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:40.821141 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:40.821143 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:40.821404 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:40.821559 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:40.821908 (+ 349us) server_negotiation.cc:299] Negotiation successful
0805 23:10:40.822036 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260805 23:10:40.822456 32741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:40.811674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:39541 (local address 127.0.0.1:60624)
0805 23:10:40.812131 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:40.812147 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:40.812373 (+ 226us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:40.812787 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:40.812790 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:40.813109 (+ 319us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:40.813294 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:40.813305 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:40.814450 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:40.814454 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:40.815144 (+ 690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:40.815153 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:40.815987 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:40.815992 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:40.816136 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:40.817516 (+ 1380us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:10:40.817534 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:40.818039 (+ 505us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:10:40.820282 (+ 2243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:40.820288 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:40.820290 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:40.820569 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:40.820911 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:40.820917 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:40.820921 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:40.821027 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:40.821568 (+ 541us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:10:40.821571 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:10:40.821670 (+ 99us) client_negotiation.cc:769] Sending connection context
0805 23:10:40.821829 (+ 159us) client_negotiation.cc:240] Negotiation successful
0805 23:10:40.821991 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":346,"mutex_wait_us":69,"thread_start_us":99,"threads_started":1}
I20260805 23:10:40.822912 32742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:40.811731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.253:38299 (local address 127.0.0.1:33826)
0805 23:10:40.812201 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:40.812216 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:40.812328 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:40.812657 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:40.812660 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:40.812971 (+ 311us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:40.813093 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:40.813100 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:40.814287 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:40.814291 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:40.815365 (+ 1074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:40.815378 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:40.816127 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:40.816130 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:40.816199 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:40.816922 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:10:40.816935 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:40.817312 (+ 377us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:10:40.819998 (+ 2686us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:40.820002 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:40.820005 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:40.820377 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:40.820721 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:40.820728 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:40.820730 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:40.820818 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:40.821328 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:10:40.821332 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:10:40.821424 (+ 92us) client_negotiation.cc:769] Sending connection context
0805 23:10:40.821687 (+ 263us) client_negotiation.cc:240] Negotiation successful
0805 23:10:40.822026 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":398,"mutex_wait_us":28,"thread_start_us":110,"threads_started":1}
Aug 05 23:10:40 dist-test-slave-ss4n krb5kdc[32127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971439, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.92.254@KRBTEST.COM for oryx/127.25.92.253@KRBTEST.COM
WARNING: no policy specified for oryx/127.25.92.252@KRBTEST.COM; defaulting to no policy
I20260805 23:10:40.839695 32721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:40.827816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33830 (local address 127.25.92.253:38299)
0805 23:10:40.827954 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:40.827959 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:40.828312 (+ 353us) server_negotiation.cc:407] Connection header received
0805 23:10:40.828700 (+ 388us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:40.828704 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:40.828768 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:40.828905 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:40.829923 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:40.830523 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:40.831529 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:40.831951 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:40.832079 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:40.835223 (+ 3144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:40.835231 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:40.835238 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:40.835279 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:40.837313 (+ 2034us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:40.837985 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:40.837989 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:40.837991 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:40.838089 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:40.838435 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:40.838443 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:40.838447 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:40.838746 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:40.838877 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:40.839310 (+ 433us) server_negotiation.cc:299] Negotiation successful
0805 23:10:40.839459 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260805 23:10:40.839707 32743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:40.827763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.253:38299 (local address 127.0.0.1:33830)
0805 23:10:40.828170 (+ 407us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:40.828224 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:40.828369 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:40.828996 (+ 627us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:40.829007 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:40.829619 (+ 612us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:40.829774 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:40.829782 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:40.830655 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:40.830660 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:40.831389 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:40.831400 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:40.832096 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:40.832100 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:40.832240 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:40.832948 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:10:40.832964 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:40.835009 (+ 2045us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:10:40.837465 (+ 2456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:40.837517 (+ 52us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:40.837526 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:40.837857 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:40.838206 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:40.838209 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:40.838211 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:40.838309 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:40.838881 (+ 572us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:10:40.838889 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:10:40.839044 (+ 155us) client_negotiation.cc:769] Sending connection context
0805 23:10:40.839200 (+ 156us) client_negotiation.cc:240] Negotiation successful
0805 23:10:40.839423 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":145,"threads_started":1}
I20260805 23:10:40.842291 32174 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.25.92.253:38299: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "d72d1f2dd8074bbbaf2fc4761feb7080"
member_type: NON_VOTER
last_known_addr {
host: "127.25.92.253"
port: 38299
}
attrs {
promote: true
}
}
dest_uuid: "87c2eb89255e45f0a2feae991859aa8e"
cas_config_opid_index: -1
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.25.92.252@KRBTEST.COM" created.
I20260805 23:10:40.843387 32174 consensus_queue.cc:237] T 00000000000000000000000000000000 P 87c2eb89255e45f0a2feae991859aa8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "87c2eb89255e45f0a2feae991859aa8e" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 39541 } } peers { permanent_uuid: "d72d1f2dd8074bbbaf2fc4761feb7080" member_type: NON_VOTER last_known_addr { host: "127.25.92.253" port: 38299 } attrs { promote: true } }
I20260805 23:10:40.846233 32749 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 87c2eb89255e45f0a2feae991859aa8e [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER d72d1f2dd8074bbbaf2fc4761feb7080 (127.25.92.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "87c2eb89255e45f0a2feae991859aa8e" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 39541 } } peers { permanent_uuid: "d72d1f2dd8074bbbaf2fc4761feb7080" member_type: NON_VOTER last_known_addr { host: "127.25.92.253" port: 38299 } attrs { promote: true } } }
I20260805 23:10:40.846585 32691 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d72d1f2dd8074bbbaf2fc4761feb7080 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:10:40.848152 32691 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d72d1f2dd8074bbbaf2fc4761feb7080 [term 1 FOLLOWER]: Refusing update from remote peer 87c2eb89255e45f0a2feae991859aa8e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260805 23:10:40.848999 32748 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 87c2eb89255e45f0a2feae991859aa8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72d1f2dd8074bbbaf2fc4761feb7080" member_type: NON_VOTER last_known_addr { host: "127.25.92.253" port: 38299 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:10:40.849731 32749 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.25.92.253:38299
I20260805 23:10:40.850241 32749 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87c2eb89255e45f0a2feae991859aa8e [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "87c2eb89255e45f0a2feae991859aa8e" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "87c2eb89255e45f0a2feae991859aa8e" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 39541 } } peers { permanent_uuid: "d72d1f2dd8074bbbaf2fc4761feb7080" member_type: NON_VOTER last_known_addr { host: "127.25.92.253" port: 38299 } attrs { promote: true } } }
I20260805 23:10:40.850332 32648 master.cc:537] Master@127.25.92.253:38299 shutting down...
I20260805 23:10:40.850464 32749 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87c2eb89255e45f0a2feae991859aa8e [sys.catalog]: This master's current role is: LEADER
W20260805 23:10:40.851529 32156 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260805 23:10:40.852723 32156 consensus_peers.cc:487] T 00000000000000000000000000000000 P 87c2eb89255e45f0a2feae991859aa8e -> Peer d72d1f2dd8074bbbaf2fc4761feb7080 (127.25.92.253:38299): Couldn't send request to peer d72d1f2dd8074bbbaf2fc4761feb7080. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260805 23:10:40.855530 32648 raft_consensus.cc:2229] T 00000000000000000000000000000000 P d72d1f2dd8074bbbaf2fc4761feb7080 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 23:10:40.855895 32648 raft_consensus.cc:2258] T 00000000000000000000000000000000 P d72d1f2dd8074bbbaf2fc4761feb7080 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:10:40.856081 32648 tablet_replica.cc:330] T 00000000000000000000000000000000 P d72d1f2dd8074bbbaf2fc4761feb7080: stopping tablet replica
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.25.92.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/krb5kdc/oryx_127.25.92.252.keytab.
Entry for principal oryx/127.25.92.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/krb5kdc/oryx_127.25.92.252.keytab.
I20260805 23:10:40.869313 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.252:41853
--webserver_interface=127.25.92.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/krb5kdc/oryx_127.25.92.252.keytab
--principal=oryx/127.25.92.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:44103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.92.254:39541,127.25.92.253:38299,127.25.92.252:41853
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260805 23:10:40.872008 32648 master.cc:552] Master@127.25.92.253:38299 shutdown complete.
I20260805 23:10:40.872078 32648 master_runner.cc:305] Clearing existing system tablet
I20260805 23:10:40.872325 32648 fs_manager.cc:359] Metadata directory not provided
I20260805 23:10:40.872380 32648 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:10:40.873787 32648 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260805 23:10:40.874753 32756 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:40.875005 32648 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:10:40.875129 32648 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260805 23:10:40.875278 32648 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/master-1/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/master-1/wal
uuid: "d72d1f2dd8074bbbaf2fc4761feb7080"
format_stamp: "Formatted at 2026-08-05 23:10:40 on dist-test-slave-ss4n"
server_key: "25e6ad65c9be854fa29b540a40c04eab"
server_key_iv: "1dc1beab376c057317a44f3d33c439c2"
server_key_version: "encryptionkey@0"
I20260805 23:10:40.877643 32648 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P d72d1f2dd8074bbbaf2fc4761feb7080: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:10:40.880069 32648 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P d72d1f2dd8074bbbaf2fc4761feb7080: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260805 23:10:40.880169 32648 log.cc:1196] T 00000000000000000000000000000000 P d72d1f2dd8074bbbaf2fc4761feb7080: Deleting WAL directory at /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260805 23:10:40.880422 32648 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P d72d1f2dd8074bbbaf2fc4761feb7080: Deleting consensus metadata
I20260805 23:10:40.880759 32648 master_runner.cc:315] Copying system tablet from 127.25.92.254:39541
I20260805 23:10:40.882316 32648 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P d72d1f2dd8074bbbaf2fc4761feb7080: tablet copy: Beginning tablet copy session from remote peer at address 127.25.92.254:39541
I20260805 23:10:40.903720 32213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:40.883183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60632 (local address 127.25.92.254:39541)
0805 23:10:40.883326 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:40.883331 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:40.893895 (+ 10564us) server_negotiation.cc:407] Connection header received
0805 23:10:40.894009 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:40.894016 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:40.894086 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:40.894209 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:40.894864 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:40.895793 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:40.897123 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:40.897596 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:40.897712 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:40.899235 (+ 1523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:40.899240 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:40.899243 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:40.899275 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:40.901218 (+ 1943us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:40.901853 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:40.901858 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:40.901860 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:40.902008 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:40.902482 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:40.902486 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:40.902488 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:40.902758 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:40.902839 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:40.903358 (+ 519us) server_negotiation.cc:299] Negotiation successful
0805 23:10:40.903511 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260805 23:10:40.904369 32763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:40.883067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:39541 (local address 127.0.0.1:60632)
0805 23:10:40.893741 (+ 10674us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:40.893760 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:40.893910 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:40.894264 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:40.894267 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:40.894531 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:40.894693 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:40.894701 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:40.895942 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:40.895947 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:40.896928 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:40.896941 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:40.897754 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:40.897759 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:40.897868 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:40.898555 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:10:40.898571 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:40.899055 (+ 484us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:10:40.901403 (+ 2348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:40.901407 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:40.901409 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:40.901730 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:40.902157 (+ 427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:40.902161 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:40.902163 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:40.902293 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:40.902923 (+ 630us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:10:40.902926 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:10:40.903106 (+ 180us) client_negotiation.cc:769] Sending connection context
0805 23:10:40.903982 (+ 876us) client_negotiation.cc:240] Negotiation successful
0805 23:10:40.904155 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10573,"thread_start_us":96,"threads_started":1}
I20260805 23:10:40.905301 32194 tablet_copy_service.cc:140] P 87c2eb89255e45f0a2feae991859aa8e: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer d72d1f2dd8074bbbaf2fc4761feb7080 ({username='oryx', principal='oryx/127.25.92.253@KRBTEST.COM'} at 127.0.0.1:60632)
I20260805 23:10:40.905666 32194 tablet_copy_service.cc:161] P 87c2eb89255e45f0a2feae991859aa8e: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer d72d1f2dd8074bbbaf2fc4761feb7080 at {username='oryx', principal='oryx/127.25.92.253@KRBTEST.COM'} at 127.0.0.1:60632: session id = d72d1f2dd8074bbbaf2fc4761feb7080-00000000000000000000000000000000
I20260805 23:10:40.907699 32194 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 87c2eb89255e45f0a2feae991859aa8e: Tablet Copy: opened 0 blocks and 1 log segments
I20260805 23:10:40.909824 32648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260805 23:10:40.912633 32648 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P d72d1f2dd8074bbbaf2fc4761feb7080: tablet copy: Starting download of 0 data blocks...
I20260805 23:10:40.912856 32648 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P d72d1f2dd8074bbbaf2fc4761feb7080: tablet copy: Starting download of 1 WAL segments...
I20260805 23:10:40.914851 32648 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P d72d1f2dd8074bbbaf2fc4761feb7080: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260805 23:10:40.917172 32194 tablet_copy_service.cc:342] P 87c2eb89255e45f0a2feae991859aa8e: Request end of tablet copy session d72d1f2dd8074bbbaf2fc4761feb7080-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.25.92.253@KRBTEST.COM'} at 127.0.0.1:60632
I20260805 23:10:40.917327 32194 tablet_copy_service.cc:433] P 87c2eb89255e45f0a2feae991859aa8e: ending tablet copy session d72d1f2dd8074bbbaf2fc4761feb7080-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer d72d1f2dd8074bbbaf2fc4761feb7080
I20260805 23:10:40.925855 32648 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:10:40.928027 32767 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:40.928068 32764 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:10:40.928844 32648 server_base.cc:1034] running on GCE node
W20260805 23:10:40.929661 32765 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:10:40.929979 32648 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:10:40.930331 32648 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:10:40.931497 32648 hybrid_clock.cc:648] HybridClock initialized: now 1785971440931481 us; error 34 us; skew 500 ppm
Aug 05 23:10:40 dist-test-slave-ss4n krb5kdc[32127](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971440, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.92.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:10:40.933403 32648 init.cc:377] Logged in from keytab as oryx/127.25.92.253@KRBTEST.COM (short username oryx)
I20260805 23:10:40.933979 32648 webserver.cc:460] Webserver started at http://127.25.92.253:44287/ using document root <none> and password file <none>
I20260805 23:10:40.934206 32648 fs_manager.cc:359] Metadata directory not provided
I20260805 23:10:40.934254 32648 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:10:40.935801 32648 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260805 23:10:40.937054 306 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:40.937412 32648 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260805 23:10:40.937527 32648 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/master-1/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/master-1/wal
uuid: "d72d1f2dd8074bbbaf2fc4761feb7080"
format_stamp: "Formatted at 2026-08-05 23:10:40 on dist-test-slave-ss4n"
server_key: "25e6ad65c9be854fa29b540a40c04eab"
server_key_iv: "1dc1beab376c057317a44f3d33c439c2"
server_key_version: "encryptionkey@0"
I20260805 23:10:40.937624 32648 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:10:40.951560 32648 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:10:40.954195 32648 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:10:40.959808 32648 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.253:38299
I20260805 23:10:40.959851 358 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.253:38299 every 8 connection(s)
I20260805 23:10:40.960466 32648 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/master-1/data/info.pb
I20260805 23:10:40.962659 359 sys_catalog.cc:263] Verifying existing consensus state
I20260805 23:10:40.963861 359 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d72d1f2dd8074bbbaf2fc4761feb7080: Bootstrap starting.
I20260805 23:10:40.976413 359 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d72d1f2dd8074bbbaf2fc4761feb7080: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260805 23:10:40.976771 359 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d72d1f2dd8074bbbaf2fc4761feb7080: Bootstrap complete.
I20260805 23:10:40.977404 359 raft_consensus.cc:348] T 00000000000000000000000000000000 P d72d1f2dd8074bbbaf2fc4761feb7080 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "87c2eb89255e45f0a2feae991859aa8e" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 39541 } } peers { permanent_uuid: "d72d1f2dd8074bbbaf2fc4761feb7080" member_type: NON_VOTER last_known_addr { host: "127.25.92.253" port: 38299 } attrs { promote: true } }
I20260805 23:10:40.977591 359 raft_consensus.cc:729] T 00000000000000000000000000000000 P d72d1f2dd8074bbbaf2fc4761feb7080 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: d72d1f2dd8074bbbaf2fc4761feb7080, State: Initialized, Role: LEARNER
I20260805 23:10:40.977705 359 consensus_queue.cc:260] T 00000000000000000000000000000000 P d72d1f2dd8074bbbaf2fc4761feb7080 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "87c2eb89255e45f0a2feae991859aa8e" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 39541 } } peers { permanent_uuid: "d72d1f2dd8074bbbaf2fc4761feb7080" member_type: NON_VOTER last_known_addr { host: "127.25.92.253" port: 38299 } attrs { promote: true } }
I20260805 23:10:40.978193 359 sys_catalog.cc:564] T 00000000000000000000000000000000 P d72d1f2dd8074bbbaf2fc4761feb7080 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:10:40.978094 362 sys_catalog.cc:455] T 00000000000000000000000000000000 P d72d1f2dd8074bbbaf2fc4761feb7080 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "87c2eb89255e45f0a2feae991859aa8e" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 39541 } } peers { permanent_uuid: "d72d1f2dd8074bbbaf2fc4761feb7080" member_type: NON_VOTER last_known_addr { host: "127.25.92.253" port: 38299 } attrs { promote: true } } }
I20260805 23:10:40.978416 362 sys_catalog.cc:458] T 00000000000000000000000000000000 P d72d1f2dd8074bbbaf2fc4761feb7080 [sys.catalog]: This master's current role is: LEARNER
I20260805 23:10:40.981752 373 catalog_manager.cc:1259] Loaded cluster ID: cfa50e6d78c6414ca13b867ff0c3f044
I20260805 23:10:40.981874 373 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d72d1f2dd8074bbbaf2fc4761feb7080: loading cluster ID for follower catalog manager: success
I20260805 23:10:40.984388 373 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d72d1f2dd8074bbbaf2fc4761feb7080: acquiring CA information for follower catalog manager: success
I20260805 23:10:40.986006 373 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d72d1f2dd8074bbbaf2fc4761feb7080: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260805 23:10:41.019224 32754 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:10:41.019507 32754 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:10:41.019563 32754 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:10:41.024258 32754 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:10:41.024331 32754 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:10:41.024355 32754 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:10:41.024374 32754 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:10:41.024391 32754 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:10:41.030835 32754 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:44103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.25.92.254:39541,127.25.92.253:38299,127.25.92.252:41853
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/krb5kdc/oryx_127.25.92.252.keytab
--principal=oryx/127.25.92.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.252:41853
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.25.92.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32754
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:10:41.031992 32754 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:10:41.033046 32754 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:10:41.042407 381 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:41.042366 379 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:41.042359 378 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:10:41.042706 32754 server_base.cc:1034] running on GCE node
I20260805 23:10:41.043466 32754 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:10:41.044680 32754 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:10:41.045867 32754 hybrid_clock.cc:648] HybridClock initialized: now 1785971441045830 us; error 50 us; skew 500 ppm
Aug 05 23:10:41 dist-test-slave-ss4n krb5kdc[32127](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971441, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.92.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:10:41.049053 32754 init.cc:377] Logged in from keytab as oryx/127.25.92.252@KRBTEST.COM (short username oryx)
I20260805 23:10:41.050076 32754 webserver.cc:460] Webserver started at http://127.25.92.252:41459/ using document root <none> and password file <none>
I20260805 23:10:41.050683 32754 fs_manager.cc:359] Metadata directory not provided
I20260805 23:10:41.050782 32754 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:10:41.051029 32754 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:10:41.053187 32754 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/master-2/data/instance:
uuid: "4cc69a74b0534df2bbc6b22b987d8f63"
format_stamp: "Formatted at 2026-08-05 23:10:41 on dist-test-slave-ss4n"
server_key: "7db528f8fc7596a791d9b56f405d24ad"
server_key_iv: "f10c2b861cea9010a35cd3f6f3726c43"
server_key_version: "encryptionkey@0"
I20260805 23:10:41.053818 32754 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/master-2/wal/instance:
uuid: "4cc69a74b0534df2bbc6b22b987d8f63"
format_stamp: "Formatted at 2026-08-05 23:10:41 on dist-test-slave-ss4n"
server_key: "7db528f8fc7596a791d9b56f405d24ad"
server_key_iv: "f10c2b861cea9010a35cd3f6f3726c43"
server_key_version: "encryptionkey@0"
I20260805 23:10:41.058025 32754 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260805 23:10:41.060753 388 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:41.062304 32754 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:10:41.062466 32754 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/master-2/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/master-2/wal
uuid: "4cc69a74b0534df2bbc6b22b987d8f63"
format_stamp: "Formatted at 2026-08-05 23:10:41 on dist-test-slave-ss4n"
server_key: "7db528f8fc7596a791d9b56f405d24ad"
server_key_iv: "f10c2b861cea9010a35cd3f6f3726c43"
server_key_version: "encryptionkey@0"
I20260805 23:10:41.062636 32754 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:10:41.079594 32369 heartbeater.cc:503] Master 127.25.92.254:39541 was elected leader, sending a full tablet report...
I20260805 23:10:41.082163 32754 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:10:41.085856 32754 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:10:41.086024 32754 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:10:41.093338 32754 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.252:41853
I20260805 23:10:41.093755 443 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.252:41853 every 8 connection(s)
I20260805 23:10:41.094453 32754 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/master-2/data/info.pb
I20260805 23:10:41.097110 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 32754
I20260805 23:10:41.097606 444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:10:41.102836 444 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.92.254" port: 39541 } has no permanent_uuid. Determining permanent_uuid...
Aug 05 23:10:41 dist-test-slave-ss4n krb5kdc[32127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971439, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.25.92.252@KRBTEST.COM
Aug 05 23:10:41 dist-test-slave-ss4n krb5kdc[32127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971441, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.92.252@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM
I20260805 23:10:41.113418 447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:41.098702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55548 (local address 127.25.92.252:41853)
0805 23:10:41.099148 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:41.099167 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:41.099211 (+ 44us) server_negotiation.cc:407] Connection header received
0805 23:10:41.099986 (+ 775us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:41.100010 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:41.100422 (+ 412us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:41.100800 (+ 378us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:41.101646 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:41.103153 (+ 1507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:41.104105 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:41.104718 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:41.104889 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:41.107831 (+ 2942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:41.107836 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:41.107847 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:41.107879 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:41.110081 (+ 2202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:41.110650 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:41.110655 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:41.110660 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:41.110857 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:41.111218 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:41.111222 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:41.111224 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:41.111668 (+ 444us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:41.111928 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:41.112213 (+ 285us) server_negotiation.cc:299] Negotiation successful
0805 23:10:41.112580 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":141,"threads_started":1}
I20260805 23:10:41.115422 32213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:41.104900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60638 (local address 127.25.92.254:39541)
0805 23:10:41.105043 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:41.105048 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:41.105477 (+ 429us) server_negotiation.cc:407] Connection header received
0805 23:10:41.105650 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:41.105654 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:41.105717 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:41.105810 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:41.106451 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:41.107074 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:41.108217 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:41.108578 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:41.108665 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:41.111144 (+ 2479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:41.111154 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:41.111157 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:41.111182 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:41.112765 (+ 1583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:41.113230 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:41.113234 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:41.113236 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:41.113319 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:41.114481 (+ 1162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:41.114484 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:41.114486 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:41.114684 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:41.114777 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:41.115202 (+ 425us) server_negotiation.cc:299] Negotiation successful
0805 23:10:41.115309 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260805 23:10:41.115437 448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:41.104714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:39541 (local address 127.0.0.1:60638)
0805 23:10:41.105327 (+ 613us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:41.105379 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:41.105532 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:41.105862 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:41.105866 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:41.106205 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:41.106325 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:41.106331 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:41.107212 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:41.107217 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:41.108062 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:41.108073 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:41.108704 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:41.108708 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:41.108867 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:41.109284 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:10:41.109298 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:41.111000 (+ 1702us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:10:41.112877 (+ 1877us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:41.112888 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:41.112894 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:41.113125 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:41.113422 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:41.113425 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:41.113427 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:41.113569 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:41.114795 (+ 1226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:10:41.114801 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:10:41.114931 (+ 130us) client_negotiation.cc:769] Sending connection context
0805 23:10:41.115076 (+ 145us) client_negotiation.cc:240] Negotiation successful
0805 23:10:41.115256 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":159,"threads_started":1}
I20260805 23:10:41.116519 444 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.92.253" port: 38299 } has no permanent_uuid. Determining permanent_uuid...
Aug 05 23:10:41 dist-test-slave-ss4n krb5kdc[32127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971441, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.92.252@KRBTEST.COM for oryx/127.25.92.253@KRBTEST.COM
I20260805 23:10:41.125908 448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:41.117091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.253:38299 (local address 127.0.0.1:33846)
0805 23:10:41.117459 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:41.117524 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:41.117599 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:41.117819 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:41.117821 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:41.117953 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:41.118061 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:41.118066 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:41.119115 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:41.119117 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:41.119687 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:41.119696 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:41.120252 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:41.120255 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:41.120329 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:41.120851 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:10:41.120863 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:41.122290 (+ 1427us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:10:41.124319 (+ 2029us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:41.124322 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:41.124323 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:41.124523 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:41.124841 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:41.124843 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:41.124844 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:41.124917 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:41.125310 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:10:41.125313 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:10:41.125406 (+ 93us) client_negotiation.cc:769] Sending connection context
0805 23:10:41.125651 (+ 245us) client_negotiation.cc:240] Negotiation successful
0805 23:10:41.125776 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":309}
I20260805 23:10:41.126003 449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:41.117249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33846 (local address 127.25.92.253:38299)
0805 23:10:41.117546 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:41.117551 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:41.117570 (+ 19us) server_negotiation.cc:407] Connection header received
0805 23:10:41.117667 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:41.117669 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:41.117710 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:41.117784 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:41.118193 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:41.118974 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:41.119806 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:41.120075 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:41.120251 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:41.122451 (+ 2200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:41.122455 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:41.122457 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:41.122483 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:41.124189 (+ 1706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:41.124672 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:41.124675 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:41.124676 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:41.124756 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:41.124994 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:41.124996 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:41.124997 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:41.125191 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:41.125268 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:41.125735 (+ 467us) server_negotiation.cc:299] Negotiation successful
0805 23:10:41.125859 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":91,"threads_started":1}
I20260805 23:10:41.127019 444 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.92.252" port: 41853 } has no permanent_uuid. Determining permanent_uuid...
Aug 05 23:10:41 dist-test-slave-ss4n krb5kdc[32127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971441, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.92.252@KRBTEST.COM for oryx/127.25.92.252@KRBTEST.COM
I20260805 23:10:41.136015 448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:41.127563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.252:41853 (local address 127.0.0.1:55560)
0805 23:10:41.127700 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:41.127715 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:41.127796 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:41.128002 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:41.128004 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:41.128135 (+ 131us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:41.128209 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:41.128214 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:41.129750 (+ 1536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:41.129753 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:41.130258 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:41.130267 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:41.130766 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:41.130769 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:41.130829 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:41.131379 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:10:41.131394 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:41.132720 (+ 1326us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:10:41.134704 (+ 1984us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:41.134707 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:41.134708 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:41.134893 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:41.135155 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:41.135157 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:41.135158 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:41.135230 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:41.135620 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:10:41.135622 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:10:41.135698 (+ 76us) client_negotiation.cc:769] Sending connection context
0805 23:10:41.135780 (+ 82us) client_negotiation.cc:240] Negotiation successful
0805 23:10:41.135884 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64}
I20260805 23:10:41.136104 447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:41.127642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55560 (local address 127.25.92.252:41853)
0805 23:10:41.127757 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:41.127760 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:41.127772 (+ 12us) server_negotiation.cc:407] Connection header received
0805 23:10:41.127870 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:41.127872 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:41.127907 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:41.127974 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:41.128372 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:41.129029 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:41.130383 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:41.130647 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:41.130747 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:41.132864 (+ 2117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:41.132867 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:41.132870 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:41.132892 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:41.134581 (+ 1689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:41.134987 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:41.134989 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:41.134990 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:41.135071 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:41.135320 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:41.135323 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:41.135324 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:41.135529 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:41.135619 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:41.135845 (+ 226us) server_negotiation.cc:299] Negotiation successful
0805 23:10:41.135955 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260805 23:10:41.139531 444 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4cc69a74b0534df2bbc6b22b987d8f63: Bootstrap starting.
I20260805 23:10:41.141897 444 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4cc69a74b0534df2bbc6b22b987d8f63: Neither blocks nor log segments found. Creating new log.
I20260805 23:10:41.142720 444 log.cc:824] T 00000000000000000000000000000000 P 4cc69a74b0534df2bbc6b22b987d8f63: Log is configured to *not* fsync() on all Append() calls
I20260805 23:10:41.144724 444 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4cc69a74b0534df2bbc6b22b987d8f63: No bootstrap required, opened a new log
I20260805 23:10:41.147846 444 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4cc69a74b0534df2bbc6b22b987d8f63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87c2eb89255e45f0a2feae991859aa8e" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 39541 } } peers { permanent_uuid: "d72d1f2dd8074bbbaf2fc4761feb7080" member_type: VOTER last_known_addr { host: "127.25.92.253" port: 38299 } } peers { permanent_uuid: "4cc69a74b0534df2bbc6b22b987d8f63" member_type: VOTER last_known_addr { host: "127.25.92.252" port: 41853 } }
I20260805 23:10:41.148056 444 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4cc69a74b0534df2bbc6b22b987d8f63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:10:41.148090 444 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4cc69a74b0534df2bbc6b22b987d8f63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cc69a74b0534df2bbc6b22b987d8f63, State: Initialized, Role: FOLLOWER
I20260805 23:10:41.148527 444 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4cc69a74b0534df2bbc6b22b987d8f63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87c2eb89255e45f0a2feae991859aa8e" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 39541 } } peers { permanent_uuid: "d72d1f2dd8074bbbaf2fc4761feb7080" member_type: VOTER last_known_addr { host: "127.25.92.253" port: 38299 } } peers { permanent_uuid: "4cc69a74b0534df2bbc6b22b987d8f63" member_type: VOTER last_known_addr { host: "127.25.92.252" port: 41853 } }
I20260805 23:10:41.149000 451 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4cc69a74b0534df2bbc6b22b987d8f63 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87c2eb89255e45f0a2feae991859aa8e" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 39541 } } peers { permanent_uuid: "d72d1f2dd8074bbbaf2fc4761feb7080" member_type: VOTER last_known_addr { host: "127.25.92.253" port: 38299 } } peers { permanent_uuid: "4cc69a74b0534df2bbc6b22b987d8f63" member_type: VOTER last_known_addr { host: "127.25.92.252" port: 41853 } } }
I20260805 23:10:41.149145 451 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4cc69a74b0534df2bbc6b22b987d8f63 [sys.catalog]: This master's current role is: FOLLOWER
I20260805 23:10:41.149787 444 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4cc69a74b0534df2bbc6b22b987d8f63 [sys.catalog]: configured and running, proceeding with master startup.
W20260805 23:10:41.155083 462 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4cc69a74b0534df2bbc6b22b987d8f63: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260805 23:10:41.155234 462 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260805 23:10:41.158891 32754 master_runner.cc:428] Detected that this master 4cc69a74b0534df2bbc6b22b987d8f63 is joining an existing cluster
I20260805 23:10:41.158969 32754 master_runner.cc:432] Initiating AddMaster RPC to add 127.25.92.252:41853
I20260805 23:10:41.168285 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 32237
I20260805 23:10:41.173260 32213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:41.161719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60652 (local address 127.25.92.254:39541)
0805 23:10:41.161975 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:41.161979 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:41.162535 (+ 556us) server_negotiation.cc:407] Connection header received
0805 23:10:41.162743 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:41.162746 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:41.162809 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:41.162868 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:41.163837 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:41.164717 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:41.165978 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:41.166393 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:41.166478 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:41.169376 (+ 2898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:41.169383 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:41.169389 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:41.169426 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:41.171205 (+ 1779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:41.171656 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:41.171663 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:41.171669 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:41.171772 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:41.172140 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:41.172147 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:41.172153 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:41.172429 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:41.172514 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:41.172957 (+ 443us) server_negotiation.cc:299] Negotiation successful
0805 23:10:41.173092 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172}
I20260805 23:10:41.173756 469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:41.161581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:39541 (local address 127.0.0.1:60652)
0805 23:10:41.162429 (+ 848us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:41.162443 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:41.162607 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:41.163033 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:41.163039 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:41.163377 (+ 338us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:41.163698 (+ 321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:41.163706 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:41.164853 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:41.164856 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:41.165857 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:41.165865 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:41.167267 (+ 1402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:41.167274 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:41.167405 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:41.167940 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:10:41.167957 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:41.169213 (+ 1256us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:10:41.171335 (+ 2122us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:41.171338 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:41.171339 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:41.171543 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:41.171892 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:41.171898 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:41.171904 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:41.172022 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:41.172600 (+ 578us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:10:41.172607 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:10:41.172731 (+ 124us) client_negotiation.cc:769] Sending connection context
0805 23:10:41.172861 (+ 130us) client_negotiation.cc:240] Negotiation successful
0805 23:10:41.173064 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":785,"mutex_wait_us":3,"thread_start_us":119,"threads_started":1}
I20260805 23:10:41.179225 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.92.193:44453
--local_ip_for_outbound_sockets=127.25.92.193
--tserver_master_addrs=127.25.92.254:39541
--webserver_port=41671
--webserver_interface=127.25.92.193
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/krb5kdc/oryx_127.25.92.193.keytab
--principal=oryx/127.25.92.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:44103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.25.92.254:39541,127.25.92.253:38299
--tserver_master_addrs=127.25.92.254:39541,127.25.92.253:38299,127.25.92.252:41853 with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 05 23:10:41 dist-test-slave-ss4n krb5kdc[32127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971439, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.92.254@KRBTEST.COM for oryx/127.25.92.252@KRBTEST.COM
I20260805 23:10:41.189791 449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:41.161847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33848 (local address 127.25.92.253:38299)
0805 23:10:41.162066 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:41.162070 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:41.162124 (+ 54us) server_negotiation.cc:407] Connection header received
0805 23:10:41.162334 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:41.162337 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:41.162392 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:41.162478 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:41.163433 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:41.164323 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:41.165695 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:41.166069 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:41.166151 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:41.167706 (+ 1555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:41.167710 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:41.167713 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:41.167743 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:41.181945 (+ 14202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:41.187685 (+ 5740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:41.187693 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:41.187697 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:41.187820 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:41.188349 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:41.188357 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:41.188360 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:41.188663 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:41.188778 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:41.189244 (+ 466us) server_negotiation.cc:299] Negotiation successful
0805 23:10:41.189529 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149}
I20260805 23:10:41.190865 470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:41.162295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.252:41853 (local address 127.0.0.1:55576)
0805 23:10:41.162618 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:41.162633 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:41.162746 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:41.163098 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:41.163103 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:41.163386 (+ 283us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:41.163551 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:41.163563 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:41.164658 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:41.164661 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:41.165692 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:41.165703 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:41.166493 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:41.166497 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:41.166601 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:41.186110 (+ 19509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:10:41.186133 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:41.186748 (+ 615us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:10:41.189370 (+ 2622us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:41.189372 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:41.189375 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:41.189645 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:41.189919 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:41.189922 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:41.189924 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:41.189992 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:41.190368 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:10:41.190371 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:10:41.190451 (+ 80us) client_negotiation.cc:769] Sending connection context
0805 23:10:41.190527 (+ 76us) client_negotiation.cc:240] Negotiation successful
0805 23:10:41.190667 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"mutex_wait_us":20,"thread_start_us":89,"threads_started":1}
I20260805 23:10:41.190848 468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:41.161526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.253:38299 (local address 127.0.0.1:33848)
0805 23:10:41.162039 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:41.162066 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:41.162216 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:41.162608 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:41.162611 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:41.163024 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:41.163233 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:41.163245 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:41.164472 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:41.164476 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:41.165543 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:41.165554 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:41.166333 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:41.166337 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:41.166434 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:41.167062 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:10:41.167077 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:41.167537 (+ 460us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:10:41.182121 (+ 14584us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:41.182125 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:41.182128 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:41.187519 (+ 5391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:41.188080 (+ 561us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:41.188087 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:41.188090 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:41.188210 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:41.188855 (+ 645us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:10:41.188862 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:10:41.188990 (+ 128us) client_negotiation.cc:769] Sending connection context
0805 23:10:41.190361 (+ 1371us) client_negotiation.cc:240] Negotiation successful
0805 23:10:41.190561 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":129,"threads_started":1}
I20260805 23:10:41.191586 447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:41.162057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55576 (local address 127.25.92.252:41853)
0805 23:10:41.162192 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:41.162197 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:41.162699 (+ 502us) server_negotiation.cc:407] Connection header received
0805 23:10:41.162837 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:41.162844 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:41.162885 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:41.162932 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:41.163685 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:41.164537 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:41.165822 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:41.166240 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:41.166317 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:41.186927 (+ 20610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:41.186933 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:41.186940 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:41.186973 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:41.189254 (+ 2281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:41.189756 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:41.189759 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:41.189761 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:41.189834 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:41.190068 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:41.190071 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:41.190073 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:41.190253 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:41.190338 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:41.191340 (+ 1002us) server_negotiation.cc:299] Negotiation successful
0805 23:10:41.191449 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":48}
I20260805 23:10:41.195592 32743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:41.181887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.252:41853 (local address 127.0.0.1:55586)
0805 23:10:41.182008 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:41.182023 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:41.182108 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:41.182772 (+ 664us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:41.182774 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:41.182940 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:41.183040 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:41.183046 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:41.184209 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:41.184213 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:41.185093 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:41.185103 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:41.185785 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:41.185789 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:41.185889 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:41.186402 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:10:41.186417 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:41.191187 (+ 4770us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:10:41.193839 (+ 2652us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:41.193845 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:41.193849 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:41.194123 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:41.194471 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:41.194474 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:41.194476 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:41.194575 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:41.195045 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:10:41.195050 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:10:41.195169 (+ 119us) client_negotiation.cc:769] Sending connection context
0805 23:10:41.195260 (+ 91us) client_negotiation.cc:240] Negotiation successful
0805 23:10:41.195425 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260805 23:10:41.196233 472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:41.182139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55586 (local address 127.25.92.252:41853)
0805 23:10:41.182460 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:41.182468 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:41.182481 (+ 13us) server_negotiation.cc:407] Connection header received
0805 23:10:41.182550 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:41.182553 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:41.182615 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:41.182717 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:41.183153 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:41.184066 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:41.185244 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:41.185638 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:41.191699 (+ 6061us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:41.191826 (+ 127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:41.191831 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:41.191835 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:41.191856 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:41.193703 (+ 1847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:41.194258 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:41.194263 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:41.194267 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:41.194359 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:41.194702 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:41.194706 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:41.194707 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:41.194918 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:41.195878 (+ 960us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:41.195953 (+ 75us) server_negotiation.cc:299] Negotiation successful
0805 23:10:41.196071 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":251,"thread_start_us":136,"threads_started":1}
I20260805 23:10:41.197021 32174 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.25.92.252:41853: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "4cc69a74b0534df2bbc6b22b987d8f63"
member_type: NON_VOTER
last_known_addr {
host: "127.25.92.252"
port: 41853
}
attrs {
promote: true
}
}
dest_uuid: "87c2eb89255e45f0a2feae991859aa8e"
cas_config_opid_index: 5
I20260805 23:10:41.197700 32174 consensus_queue.cc:237] T 00000000000000000000000000000000 P 87c2eb89255e45f0a2feae991859aa8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "87c2eb89255e45f0a2feae991859aa8e" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 39541 } } peers { permanent_uuid: "d72d1f2dd8074bbbaf2fc4761feb7080" member_type: NON_VOTER last_known_addr { host: "127.25.92.253" port: 38299 } attrs { promote: true } } peers { permanent_uuid: "4cc69a74b0534df2bbc6b22b987d8f63" member_type: NON_VOTER last_known_addr { host: "127.25.92.252" port: 41853 } attrs { promote: true } }
I20260805 23:10:41.199888 417 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4cc69a74b0534df2bbc6b22b987d8f63 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:10:41.199806 32751 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 87c2eb89255e45f0a2feae991859aa8e [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 4cc69a74b0534df2bbc6b22b987d8f63 (127.25.92.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "87c2eb89255e45f0a2feae991859aa8e" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 39541 } } peers { permanent_uuid: "d72d1f2dd8074bbbaf2fc4761feb7080" member_type: NON_VOTER last_known_addr { host: "127.25.92.253" port: 38299 } attrs { promote: true } } peers { permanent_uuid: "4cc69a74b0534df2bbc6b22b987d8f63" member_type: NON_VOTER last_known_addr { host: "127.25.92.252" port: 41853 } attrs { promote: true } } }
I20260805 23:10:41.201357 32751 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.25.92.252:41853
I20260805 23:10:41.201413 417 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4cc69a74b0534df2bbc6b22b987d8f63 [term 1 FOLLOWER]: Refusing update from remote peer 87c2eb89255e45f0a2feae991859aa8e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260805 23:10:41.202081 32748 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 87c2eb89255e45f0a2feae991859aa8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "4cc69a74b0534df2bbc6b22b987d8f63" member_type: NON_VOTER last_known_addr { host: "127.25.92.252" port: 41853 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:10:41.202163 32754 master.cc:537] Master@127.25.92.252:41853 shutting down...
I20260805 23:10:41.203850 32750 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87c2eb89255e45f0a2feae991859aa8e [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "87c2eb89255e45f0a2feae991859aa8e" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "87c2eb89255e45f0a2feae991859aa8e" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 39541 } } peers { permanent_uuid: "d72d1f2dd8074bbbaf2fc4761feb7080" member_type: NON_VOTER last_known_addr { host: "127.25.92.253" port: 38299 } attrs { promote: true } } peers { permanent_uuid: "4cc69a74b0534df2bbc6b22b987d8f63" member_type: NON_VOTER last_known_addr { host: "127.25.92.252" port: 41853 } attrs { promote: true } } }
I20260805 23:10:41.203985 32750 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87c2eb89255e45f0a2feae991859aa8e [sys.catalog]: This master's current role is: LEADER
W20260805 23:10:41.204285 32156 consensus_peers.cc:487] T 00000000000000000000000000000000 P 87c2eb89255e45f0a2feae991859aa8e -> Peer 4cc69a74b0534df2bbc6b22b987d8f63 (127.25.92.252:41853): Couldn't send request to peer 4cc69a74b0534df2bbc6b22b987d8f63. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260805 23:10:41.209024 32754 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 4cc69a74b0534df2bbc6b22b987d8f63 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 23:10:41.209362 32754 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 4cc69a74b0534df2bbc6b22b987d8f63 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:10:41.209533 32754 tablet_replica.cc:330] T 00000000000000000000000000000000 P 4cc69a74b0534df2bbc6b22b987d8f63: stopping tablet replica
I20260805 23:10:41.209838 32743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:41.199020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.253:38299 (local address 127.0.0.1:33856)
0805 23:10:41.199552 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:41.199569 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:41.199647 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:41.203215 (+ 3568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:41.203221 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:41.203245 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:10:41.203380 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:41.203390 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:41.206224 (+ 2834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:41.206228 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:41.207510 (+ 1282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:41.207523 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:41.209290 (+ 1767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:41.209295 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:41.209405 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:41.209460 (+ 55us) client_negotiation.cc:769] Sending connection context
0805 23:10:41.209581 (+ 121us) client_negotiation.cc:240] Negotiation successful
0805 23:10:41.209690 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":442}
I20260805 23:10:41.210345 449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:41.199259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33856 (local address 127.25.92.253:38299)
0805 23:10:41.202847 (+ 3588us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:41.202852 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:41.202867 (+ 15us) server_negotiation.cc:407] Connection header received
0805 23:10:41.202918 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:41.202921 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:41.202982 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:41.203076 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:10:41.204135 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:41.204983 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:41.207944 (+ 2961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:41.208740 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:41.209054 (+ 314us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:41.209220 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:41.210138 (+ 918us) server_negotiation.cc:299] Negotiation successful
0805 23:10:41.210229 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3494}
I20260805 23:10:41.210824 333 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d72d1f2dd8074bbbaf2fc4761feb7080 [term 1 LEARNER]: Refusing update from remote peer 87c2eb89255e45f0a2feae991859aa8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260805 23:10:41.211283 32750 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 87c2eb89255e45f0a2feae991859aa8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72d1f2dd8074bbbaf2fc4761feb7080" member_type: NON_VOTER last_known_addr { host: "127.25.92.253" port: 38299 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260805 23:10:41.213424 333 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d72d1f2dd8074bbbaf2fc4761feb7080 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 4cc69a74b0534df2bbc6b22b987d8f63 (127.25.92.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "87c2eb89255e45f0a2feae991859aa8e" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 39541 } } peers { permanent_uuid: "d72d1f2dd8074bbbaf2fc4761feb7080" member_type: NON_VOTER last_known_addr { host: "127.25.92.253" port: 38299 } attrs { promote: true } } peers { permanent_uuid: "4cc69a74b0534df2bbc6b22b987d8f63" member_type: NON_VOTER last_known_addr { host: "127.25.92.252" port: 41853 } attrs { promote: true } } }
I20260805 23:10:41.215445 474 sys_catalog.cc:455] T 00000000000000000000000000000000 P d72d1f2dd8074bbbaf2fc4761feb7080 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "87c2eb89255e45f0a2feae991859aa8e" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "87c2eb89255e45f0a2feae991859aa8e" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 39541 } } peers { permanent_uuid: "d72d1f2dd8074bbbaf2fc4761feb7080" member_type: NON_VOTER last_known_addr { host: "127.25.92.253" port: 38299 } attrs { promote: true } } peers { permanent_uuid: "4cc69a74b0534df2bbc6b22b987d8f63" member_type: NON_VOTER last_known_addr { host: "127.25.92.252" port: 41853 } attrs { promote: true } } }
I20260805 23:10:41.215662 474 sys_catalog.cc:458] T 00000000000000000000000000000000 P d72d1f2dd8074bbbaf2fc4761feb7080 [sys.catalog]: This master's current role is: LEARNER
I20260805 23:10:41.215837 362 sys_catalog.cc:455] T 00000000000000000000000000000000 P d72d1f2dd8074bbbaf2fc4761feb7080 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 87c2eb89255e45f0a2feae991859aa8e. Latest consensus state: current_term: 1 leader_uuid: "87c2eb89255e45f0a2feae991859aa8e" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "87c2eb89255e45f0a2feae991859aa8e" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 39541 } } peers { permanent_uuid: "d72d1f2dd8074bbbaf2fc4761feb7080" member_type: NON_VOTER last_known_addr { host: "127.25.92.253" port: 38299 } attrs { promote: true } } peers { permanent_uuid: "4cc69a74b0534df2bbc6b22b987d8f63" member_type: NON_VOTER last_known_addr { host: "127.25.92.252" port: 41853 } attrs { promote: true } } }
I20260805 23:10:41.215960 362 sys_catalog.cc:458] T 00000000000000000000000000000000 P d72d1f2dd8074bbbaf2fc4761feb7080 [sys.catalog]: This master's current role is: LEARNER
I20260805 23:10:41.217206 32754 master.cc:552] Master@127.25.92.252:41853 shutdown complete.
I20260805 23:10:41.217267 32754 master_runner.cc:305] Clearing existing system tablet
I20260805 23:10:41.217562 32754 fs_manager.cc:359] Metadata directory not provided
I20260805 23:10:41.217628 32754 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:10:41.219129 32754 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260805 23:10:41.220089 476 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:41.220383 32754 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:10:41.220520 32754 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260805 23:10:41.220566 32754 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/master-2/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/master-2/wal
uuid: "4cc69a74b0534df2bbc6b22b987d8f63"
format_stamp: "Formatted at 2026-08-05 23:10:41 on dist-test-slave-ss4n"
server_key: "7db528f8fc7596a791d9b56f405d24ad"
server_key_iv: "f10c2b861cea9010a35cd3f6f3726c43"
server_key_version: "encryptionkey@0"
I20260805 23:10:41.222774 32754 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 4cc69a74b0534df2bbc6b22b987d8f63: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:10:41.225451 32754 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 4cc69a74b0534df2bbc6b22b987d8f63: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260805 23:10:41.225594 32754 log.cc:1196] T 00000000000000000000000000000000 P 4cc69a74b0534df2bbc6b22b987d8f63: Deleting WAL directory at /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260805 23:10:41.225873 32754 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 4cc69a74b0534df2bbc6b22b987d8f63: Deleting consensus metadata
I20260805 23:10:41.226092 32754 master_runner.cc:315] Copying system tablet from 127.25.92.254:39541
I20260805 23:10:41.227221 32750 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 87c2eb89255e45f0a2feae991859aa8e: attempting to promote NON_VOTER d72d1f2dd8074bbbaf2fc4761feb7080 to VOTER
I20260805 23:10:41.227451 32754 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 4cc69a74b0534df2bbc6b22b987d8f63: tablet copy: Beginning tablet copy session from remote peer at address 127.25.92.254:39541
I20260805 23:10:41.227761 32750 consensus_queue.cc:237] T 00000000000000000000000000000000 P 87c2eb89255e45f0a2feae991859aa8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "87c2eb89255e45f0a2feae991859aa8e" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 39541 } } peers { permanent_uuid: "d72d1f2dd8074bbbaf2fc4761feb7080" member_type: VOTER last_known_addr { host: "127.25.92.253" port: 38299 } attrs { promote: false } } peers { permanent_uuid: "4cc69a74b0534df2bbc6b22b987d8f63" member_type: NON_VOTER last_known_addr { host: "127.25.92.252" port: 41853 } attrs { promote: true } }
I20260805 23:10:41.229239 32743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:41.228860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.252:41853 (local address 127.0.0.1:55590)
0805 23:10:41.229011 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:41.229123 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.252:41853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260805 23:10:41.229404 333 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d72d1f2dd8074bbbaf2fc4761feb7080 [term 1 LEARNER]: Refusing update from remote peer 87c2eb89255e45f0a2feae991859aa8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260805 23:10:41.230103 32749 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 87c2eb89255e45f0a2feae991859aa8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72d1f2dd8074bbbaf2fc4761feb7080" member_type: VOTER last_known_addr { host: "127.25.92.253" port: 38299 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260805 23:10:41.230728 32743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:41.230328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.252:41853 (local address 127.0.0.1:55602)
0805 23:10:41.230600 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:41.230652 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.252:41853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201}
W20260805 23:10:41.230914 32156 consensus_peers.cc:487] T 00000000000000000000000000000000 P 87c2eb89255e45f0a2feae991859aa8e -> Peer 4cc69a74b0534df2bbc6b22b987d8f63 (127.25.92.252:41853): Couldn't send request to peer 4cc69a74b0534df2bbc6b22b987d8f63. Status: Network error: Client connection negotiation failed: client connection to 127.25.92.252:41853: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260805 23:10:41.232056 32749 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 87c2eb89255e45f0a2feae991859aa8e [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, d72d1f2dd8074bbbaf2fc4761feb7080 (127.25.92.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "87c2eb89255e45f0a2feae991859aa8e" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 39541 } } peers { permanent_uuid: "d72d1f2dd8074bbbaf2fc4761feb7080" member_type: VOTER last_known_addr { host: "127.25.92.253" port: 38299 } attrs { promote: false } } peers { permanent_uuid: "4cc69a74b0534df2bbc6b22b987d8f63" member_type: NON_VOTER last_known_addr { host: "127.25.92.252" port: 41853 } attrs { promote: true } } }
I20260805 23:10:41.233554 32749 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87c2eb89255e45f0a2feae991859aa8e [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "87c2eb89255e45f0a2feae991859aa8e" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "87c2eb89255e45f0a2feae991859aa8e" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 39541 } } peers { permanent_uuid: "d72d1f2dd8074bbbaf2fc4761feb7080" member_type: VOTER last_known_addr { host: "127.25.92.253" port: 38299 } attrs { promote: false } } peers { permanent_uuid: "4cc69a74b0534df2bbc6b22b987d8f63" member_type: NON_VOTER last_known_addr { host: "127.25.92.252" port: 41853 } attrs { promote: true } } }
I20260805 23:10:41.233668 32749 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87c2eb89255e45f0a2feae991859aa8e [sys.catalog]: This master's current role is: LEADER
I20260805 23:10:41.233791 32749 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87c2eb89255e45f0a2feae991859aa8e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "87c2eb89255e45f0a2feae991859aa8e" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "87c2eb89255e45f0a2feae991859aa8e" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 39541 } } peers { permanent_uuid: "d72d1f2dd8074bbbaf2fc4761feb7080" member_type: VOTER last_known_addr { host: "127.25.92.253" port: 38299 } attrs { promote: false } } peers { permanent_uuid: "4cc69a74b0534df2bbc6b22b987d8f63" member_type: NON_VOTER last_known_addr { host: "127.25.92.252" port: 41853 } attrs { promote: true } } }
I20260805 23:10:41.233857 32749 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87c2eb89255e45f0a2feae991859aa8e [sys.catalog]: This master's current role is: LEADER
I20260805 23:10:41.234704 333 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d72d1f2dd8074bbbaf2fc4761feb7080 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, d72d1f2dd8074bbbaf2fc4761feb7080 (127.25.92.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "87c2eb89255e45f0a2feae991859aa8e" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 39541 } } peers { permanent_uuid: "d72d1f2dd8074bbbaf2fc4761feb7080" member_type: VOTER last_known_addr { host: "127.25.92.253" port: 38299 } attrs { promote: false } } peers { permanent_uuid: "4cc69a74b0534df2bbc6b22b987d8f63" member_type: NON_VOTER last_known_addr { host: "127.25.92.252" port: 41853 } attrs { promote: true } } }
I20260805 23:10:41.236690 362 sys_catalog.cc:455] T 00000000000000000000000000000000 P d72d1f2dd8074bbbaf2fc4761feb7080 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "87c2eb89255e45f0a2feae991859aa8e" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "87c2eb89255e45f0a2feae991859aa8e" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 39541 } } peers { permanent_uuid: "d72d1f2dd8074bbbaf2fc4761feb7080" member_type: VOTER last_known_addr { host: "127.25.92.253" port: 38299 } attrs { promote: false } } peers { permanent_uuid: "4cc69a74b0534df2bbc6b22b987d8f63" member_type: NON_VOTER last_known_addr { host: "127.25.92.252" port: 41853 } attrs { promote: true } } }
I20260805 23:10:41.236804 362 sys_catalog.cc:458] T 00000000000000000000000000000000 P d72d1f2dd8074bbbaf2fc4761feb7080 [sys.catalog]: This master's current role is: FOLLOWER
I20260805 23:10:41.241739 482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:41.231301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:39541 (local address 127.0.0.1:60654)
0805 23:10:41.231920 (+ 619us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:41.231936 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:41.232117 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:41.232511 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:41.232514 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:41.232763 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:41.232904 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:41.232912 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:41.234172 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:41.234176 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:41.235180 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:41.235192 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:41.235951 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:41.235955 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:41.236076 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:41.236678 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:10:41.236694 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:41.237169 (+ 475us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:10:41.239926 (+ 2757us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:41.239931 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:41.239933 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:41.240218 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:41.240532 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:41.240537 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:41.240541 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:41.240650 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:41.241106 (+ 456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:10:41.241109 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:10:41.241219 (+ 110us) client_negotiation.cc:769] Sending connection context
0805 23:10:41.241337 (+ 118us) client_negotiation.cc:240] Negotiation successful
0805 23:10:41.241527 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":542,"thread_start_us":219,"threads_started":1}
I20260805 23:10:41.241783 32213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:41.231350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60654 (local address 127.25.92.254:39541)
0805 23:10:41.231465 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:41.231470 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:41.232021 (+ 551us) server_negotiation.cc:407] Connection header received
0805 23:10:41.232237 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:41.232240 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:41.232292 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:41.232347 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:41.233069 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:41.233996 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:41.235339 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:41.235691 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:41.235772 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:41.237373 (+ 1601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:41.237377 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:41.237380 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:41.237406 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:41.239803 (+ 2397us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:41.240330 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:41.240333 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:41.240335 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:41.240424 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:41.240757 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:41.240760 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:41.240762 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:41.240996 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:41.241097 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:41.241405 (+ 308us) server_negotiation.cc:299] Negotiation successful
0805 23:10:41.241620 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260805 23:10:41.242429 32194 tablet_copy_service.cc:140] P 87c2eb89255e45f0a2feae991859aa8e: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 4cc69a74b0534df2bbc6b22b987d8f63 ({username='oryx', principal='oryx/127.25.92.252@KRBTEST.COM'} at 127.0.0.1:60654)
I20260805 23:10:41.242522 32194 tablet_copy_service.cc:161] P 87c2eb89255e45f0a2feae991859aa8e: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 4cc69a74b0534df2bbc6b22b987d8f63 at {username='oryx', principal='oryx/127.25.92.252@KRBTEST.COM'} at 127.0.0.1:60654: session id = 4cc69a74b0534df2bbc6b22b987d8f63-00000000000000000000000000000000
I20260805 23:10:41.243608 32194 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 87c2eb89255e45f0a2feae991859aa8e: Tablet Copy: opened 0 blocks and 1 log segments
I20260805 23:10:41.245028 32754 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260805 23:10:41.247507 32754 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 4cc69a74b0534df2bbc6b22b987d8f63: tablet copy: Starting download of 0 data blocks...
I20260805 23:10:41.247705 32754 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 4cc69a74b0534df2bbc6b22b987d8f63: tablet copy: Starting download of 1 WAL segments...
I20260805 23:10:41.249363 32754 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 4cc69a74b0534df2bbc6b22b987d8f63: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260805 23:10:41.251595 32194 tablet_copy_service.cc:342] P 87c2eb89255e45f0a2feae991859aa8e: Request end of tablet copy session 4cc69a74b0534df2bbc6b22b987d8f63-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.25.92.252@KRBTEST.COM'} at 127.0.0.1:60654
I20260805 23:10:41.251703 32194 tablet_copy_service.cc:433] P 87c2eb89255e45f0a2feae991859aa8e: ending tablet copy session 4cc69a74b0534df2bbc6b22b987d8f63-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 4cc69a74b0534df2bbc6b22b987d8f63
I20260805 23:10:41.259120 32754 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:10:41.261243 483 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:10:41.262812 32754 server_base.cc:1034] running on GCE node
W20260805 23:10:41.262784 486 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:41.266018 484 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:10:41.266310 32754 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:10:41.266633 32754 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:10:41.267786 32754 hybrid_clock.cc:648] HybridClock initialized: now 1785971441267765 us; error 44 us; skew 500 ppm
Aug 05 23:10:41 dist-test-slave-ss4n krb5kdc[32127](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971441, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.92.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:10:41.269582 32754 init.cc:377] Logged in from keytab as oryx/127.25.92.252@KRBTEST.COM (short username oryx)
I20260805 23:10:41.270139 32754 webserver.cc:460] Webserver started at http://127.25.92.252:33615/ using document root <none> and password file <none>
I20260805 23:10:41.270354 32754 fs_manager.cc:359] Metadata directory not provided
I20260805 23:10:41.270401 32754 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:10:41.271941 32754 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260805 23:10:41.273100 493 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:41.273368 32754 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260805 23:10:41.273453 32754 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/master-2/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/master-2/wal
uuid: "4cc69a74b0534df2bbc6b22b987d8f63"
format_stamp: "Formatted at 2026-08-05 23:10:41 on dist-test-slave-ss4n"
server_key: "7db528f8fc7596a791d9b56f405d24ad"
server_key_iv: "f10c2b861cea9010a35cd3f6f3726c43"
server_key_version: "encryptionkey@0"
I20260805 23:10:41.273599 32754 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:10:41.283561 32754 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:10:41.286235 32754 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:10:41.291618 32754 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.252:41853
I20260805 23:10:41.292307 32754 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/master-2/data/info.pb
I20260805 23:10:41.294170 548 sys_catalog.cc:263] Verifying existing consensus state
I20260805 23:10:41.295313 548 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4cc69a74b0534df2bbc6b22b987d8f63: Bootstrap starting.
I20260805 23:10:41.297711 547 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.252:41853 every 8 connection(s)
I20260805 23:10:41.305759 32505 heartbeater.cc:503] Master 127.25.92.254:39541 was elected leader, sending a full tablet report...
I20260805 23:10:41.307399 548 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4cc69a74b0534df2bbc6b22b987d8f63: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260805 23:10:41.307737 548 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4cc69a74b0534df2bbc6b22b987d8f63: Bootstrap complete.
I20260805 23:10:41.308413 548 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4cc69a74b0534df2bbc6b22b987d8f63 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "87c2eb89255e45f0a2feae991859aa8e" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 39541 } } peers { permanent_uuid: "d72d1f2dd8074bbbaf2fc4761feb7080" member_type: VOTER last_known_addr { host: "127.25.92.253" port: 38299 } attrs { promote: false } } peers { permanent_uuid: "4cc69a74b0534df2bbc6b22b987d8f63" member_type: NON_VOTER last_known_addr { host: "127.25.92.252" port: 41853 } attrs { promote: true } }
I20260805 23:10:41.308564 548 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4cc69a74b0534df2bbc6b22b987d8f63 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 4cc69a74b0534df2bbc6b22b987d8f63, State: Initialized, Role: LEARNER
I20260805 23:10:41.308671 548 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4cc69a74b0534df2bbc6b22b987d8f63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "87c2eb89255e45f0a2feae991859aa8e" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 39541 } } peers { permanent_uuid: "d72d1f2dd8074bbbaf2fc4761feb7080" member_type: VOTER last_known_addr { host: "127.25.92.253" port: 38299 } attrs { promote: false } } peers { permanent_uuid: "4cc69a74b0534df2bbc6b22b987d8f63" member_type: NON_VOTER last_known_addr { host: "127.25.92.252" port: 41853 } attrs { promote: true } }
I20260805 23:10:41.309106 551 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4cc69a74b0534df2bbc6b22b987d8f63 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "87c2eb89255e45f0a2feae991859aa8e" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 39541 } } peers { permanent_uuid: "d72d1f2dd8074bbbaf2fc4761feb7080" member_type: VOTER last_known_addr { host: "127.25.92.253" port: 38299 } attrs { promote: false } } peers { permanent_uuid: "4cc69a74b0534df2bbc6b22b987d8f63" member_type: NON_VOTER last_known_addr { host: "127.25.92.252" port: 41853 } attrs { promote: true } } }
I20260805 23:10:41.309209 548 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4cc69a74b0534df2bbc6b22b987d8f63 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:10:41.309322 551 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4cc69a74b0534df2bbc6b22b987d8f63 [sys.catalog]: This master's current role is: LEARNER
I20260805 23:10:41.313841 562 catalog_manager.cc:1259] Loaded cluster ID: cfa50e6d78c6414ca13b867ff0c3f044
I20260805 23:10:41.314059 562 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 4cc69a74b0534df2bbc6b22b987d8f63: loading cluster ID for follower catalog manager: success
I20260805 23:10:41.316635 562 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 4cc69a74b0534df2bbc6b22b987d8f63: acquiring CA information for follower catalog manager: success
I20260805 23:10:41.318342 562 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 4cc69a74b0534df2bbc6b22b987d8f63: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260805 23:10:41.342099 471 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:10:41.342432 471 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:10:41.342520 471 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:10:41.347476 471 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:10:41.347568 471 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:10:41.347674 471 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.92.193
I20260805 23:10:41.354005 471 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:44103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/krb5kdc/oryx_127.25.92.193.keytab
--principal=oryx/127.25.92.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.92.193:44453
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.92.193
--webserver_port=41671
--tserver_master_addrs=127.25.92.254:39541,127.25.92.253:38299,127.25.92.252:41853
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.471
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.92.193
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:10:41.355482 471 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:10:41.356842 471 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:10:41.367479 570 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:41.367568 567 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:41.367480 568 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:10:41.367848 471 server_base.cc:1034] running on GCE node
I20260805 23:10:41.368469 471 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:10:41.369259 471 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:10:41.370461 471 hybrid_clock.cc:648] HybridClock initialized: now 1785971441370419 us; error 92 us; skew 500 ppm
Aug 05 23:10:41 dist-test-slave-ss4n krb5kdc[32127](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971441, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.92.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:10:41.373942 471 init.cc:377] Logged in from keytab as oryx/127.25.92.193@KRBTEST.COM (short username oryx)
I20260805 23:10:41.375337 471 webserver.cc:460] Webserver started at http://127.25.92.193:41671/ using document root <none> and password file <none>
I20260805 23:10:41.376019 471 fs_manager.cc:359] Metadata directory not provided
I20260805 23:10:41.376111 471 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:10:41.380426 471 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.001s
I20260805 23:10:41.382721 578 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:41.384099 471 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:10:41.384202 471 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/ts-0/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/ts-0/wal
uuid: "64c076108dc34bc68cacff2bd1d9282f"
format_stamp: "Formatted at 2026-08-05 23:10:39 on dist-test-slave-ss4n"
server_key: "8dcd60f6e0e26e78d0446e031625279c"
server_key_iv: "b6bc2c49fca0000cabdd416f8e39766e"
server_key_version: "encryptionkey@0"
I20260805 23:10:41.384763 471 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:10:41.408181 471 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:10:41.412019 471 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:10:41.412191 471 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:10:41.412799 471 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:10:41.413841 471 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:10:41.413893 471 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:41.413935 471 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:10:41.413995 471 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:41.423496 471 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.193:44453
I20260805 23:10:41.423597 693 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.193:44453 every 8 connection(s)
I20260805 23:10:41.424544 471 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/ts-0/data/info.pb
I20260805 23:10:41.429965 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 471
I20260805 23:10:41.430195 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 32374
Aug 05 23:10:41 dist-test-slave-ss4n krb5kdc[32127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971441, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.92.193@KRBTEST.COM for oryx/127.25.92.253@KRBTEST.COM
Aug 05 23:10:41 dist-test-slave-ss4n krb5kdc[32127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971441, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.92.193@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM
Aug 05 23:10:41 dist-test-slave-ss4n krb5kdc[32127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971441, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.92.193@KRBTEST.COM for oryx/127.25.92.252@KRBTEST.COM
I20260805 23:10:41.439435 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.92.194:36939
--local_ip_for_outbound_sockets=127.25.92.194
--tserver_master_addrs=127.25.92.254:39541
--webserver_port=44929
--webserver_interface=127.25.92.194
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/krb5kdc/oryx_127.25.92.194.keytab
--principal=oryx/127.25.92.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:44103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.25.92.254:39541,127.25.92.253:38299
--tserver_master_addrs=127.25.92.254:39541,127.25.92.253:38299,127.25.92.252:41853 with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260805 23:10:41.445508 698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:41.426645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.193:55605 (local address 127.25.92.252:41853)
0805 23:10:41.426953 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:41.426958 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:41.427696 (+ 738us) server_negotiation.cc:407] Connection header received
0805 23:10:41.428927 (+ 1231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:41.428931 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:41.428997 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:41.429108 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:41.431272 (+ 2164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:41.432419 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:41.434450 (+ 2031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:41.434829 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:41.434899 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:41.440438 (+ 5539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:41.440442 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:41.440445 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:41.440476 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:41.442935 (+ 2459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:41.443503 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:41.443507 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:41.443508 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:41.443596 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:41.443969 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:41.443973 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:41.443975 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:41.444216 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:41.444302 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:41.445150 (+ 848us) server_negotiation.cc:299] Negotiation successful
0805 23:10:41.445296 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":210,"thread_start_us":92,"threads_started":1}
I20260805 23:10:41.445472 32213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:41.426477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.193:40067 (local address 127.25.92.254:39541)
0805 23:10:41.426903 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:41.426908 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:41.427623 (+ 715us) server_negotiation.cc:407] Connection header received
0805 23:10:41.429046 (+ 1423us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:41.429048 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:41.429125 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:41.429211 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:41.431294 (+ 2083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:41.432274 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:41.434483 (+ 2209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:41.434960 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:41.435077 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:41.440460 (+ 5383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:41.440467 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:41.440484 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:41.440535 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:41.442785 (+ 2250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:41.443469 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:41.443475 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:41.443480 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:41.443606 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:41.443969 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:41.443973 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:41.443975 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:41.444216 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:41.444302 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:41.445150 (+ 848us) server_negotiation.cc:299] Negotiation successful
0805 23:10:41.445294 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323}
I20260805 23:10:41.448073 449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:41.428103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.193:33215 (local address 127.25.92.253:38299)
0805 23:10:41.428215 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:41.428219 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:41.428372 (+ 153us) server_negotiation.cc:407] Connection header received
0805 23:10:41.429367 (+ 995us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:41.429371 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:41.429441 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:41.429561 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:41.432187 (+ 2626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:41.432941 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:41.434450 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:41.434808 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:41.434893 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:41.440438 (+ 5545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:41.440442 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:41.440445 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:41.440476 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:41.442336 (+ 1860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:41.443325 (+ 989us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:41.443331 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:41.443339 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:41.443471 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:41.443996 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:41.444003 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:41.444007 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:41.444300 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:41.444413 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:41.447646 (+ 3233us) server_negotiation.cc:299] Negotiation successful
0805 23:10:41.447813 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260805 23:10:41.448377 699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:41.426865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.252:41853 (local address 127.25.92.193:55605)
0805 23:10:41.427587 (+ 722us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:41.427602 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:41.428676 (+ 1074us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:41.429323 (+ 647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:41.429333 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:41.430217 (+ 884us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:41.431097 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:41.431108 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:41.432570 (+ 1462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:41.432573 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:41.434305 (+ 1732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:41.434317 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:41.435108 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:41.435112 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:41.435417 (+ 305us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:41.436522 (+ 1105us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:10:41.436540 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:41.440203 (+ 3663us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:10:41.443072 (+ 2869us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:41.443075 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:41.443077 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:41.443391 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:41.443757 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:41.443760 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:41.443761 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:41.443864 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:41.444377 (+ 513us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:10:41.444385 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:10:41.444843 (+ 458us) client_negotiation.cc:769] Sending connection context
0805 23:10:41.445799 (+ 956us) client_negotiation.cc:240] Negotiation successful
0805 23:10:41.445957 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":626,"mutex_wait_us":382,"spinlock_wait_cycles":24832,"thread_start_us":153,"threads_started":1}
I20260805 23:10:41.448734 701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:41.427976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.253:38299 (local address 127.25.92.193:33215)
0805 23:10:41.428276 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:41.428293 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:41.428676 (+ 383us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:41.429570 (+ 894us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:41.429573 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:41.430181 (+ 608us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:41.432036 (+ 1855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:41.432044 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:41.433096 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:41.433099 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:41.434318 (+ 1219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:41.434324 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:41.435108 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:41.435112 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:41.435417 (+ 305us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:41.436228 (+ 811us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:10:41.436255 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:41.440203 (+ 3948us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:10:41.442481 (+ 2278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:41.442500 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:41.442620 (+ 120us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:41.443151 (+ 531us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:41.443604 (+ 453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:41.443609 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:41.443612 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:41.443841 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:41.444478 (+ 637us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:10:41.444485 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:10:41.444834 (+ 349us) client_negotiation.cc:769] Sending connection context
0805 23:10:41.445083 (+ 249us) client_negotiation.cc:240] Negotiation successful
0805 23:10:41.445557 (+ 474us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"mutex_wait_us":1853,"spinlock_wait_cycles":19200,"thread_start_us":119,"threads_started":1}
I20260805 23:10:41.449009 700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:41.426866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:39541 (local address 127.25.92.193:40067)
0805 23:10:41.427450 (+ 584us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:41.427506 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:41.428681 (+ 1175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:41.429321 (+ 640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:41.429331 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:41.430204 (+ 873us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:41.431061 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:41.431094 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:41.432464 (+ 1370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:41.432471 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:41.434322 (+ 1851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:41.434337 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:41.435410 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:41.435418 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:41.435611 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:41.436366 (+ 755us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:10:41.436389 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:41.440206 (+ 3817us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:10:41.442927 (+ 2721us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:41.442930 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:41.442932 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:41.443332 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:41.443742 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:41.443745 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:41.443749 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:41.443853 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:41.444377 (+ 524us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:10:41.444385 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:10:41.444845 (+ 460us) client_negotiation.cc:769] Sending connection context
0805 23:10:41.445889 (+ 1044us) client_negotiation.cc:240] Negotiation successful
0805 23:10:41.446012 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":355,"mutex_wait_us":284,"spinlock_wait_cycles":14976,"thread_start_us":132,"threads_started":1}
I20260805 23:10:41.450214 694 heartbeater.cc:347] Connected to a master server at 127.25.92.252:41853
I20260805 23:10:41.450294 696 heartbeater.cc:347] Connected to a master server at 127.25.92.253:38299
I20260805 23:10:41.450541 694 heartbeater.cc:464] Registering TS with master...
I20260805 23:10:41.450577 695 heartbeater.cc:347] Connected to a master server at 127.25.92.254:39541
I20260805 23:10:41.450541 696 heartbeater.cc:464] Registering TS with master...
I20260805 23:10:41.450721 695 heartbeater.cc:464] Registering TS with master...
I20260805 23:10:41.451290 694 heartbeater.cc:511] Master 127.25.92.252:41853 requested a full tablet report, sending...
I20260805 23:10:41.451308 695 heartbeater.cc:511] Master 127.25.92.254:39541 requested a full tablet report, sending...
I20260805 23:10:41.452800 32174 ts_manager.cc:194] Re-registered known tserver with Master: 64c076108dc34bc68cacff2bd1d9282f (127.25.92.193:44453)
I20260805 23:10:41.452994 510 ts_manager.cc:194] Registered new tserver with Master: 64c076108dc34bc68cacff2bd1d9282f (127.25.92.193:44453)
I20260805 23:10:41.453806 32174 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.25.92.193@KRBTEST.COM'} at 127.25.92.193:40067
I20260805 23:10:41.451290 696 heartbeater.cc:511] Master 127.25.92.253:38299 requested a full tablet report, sending...
I20260805 23:10:41.455250 323 ts_manager.cc:194] Registered new tserver with Master: 64c076108dc34bc68cacff2bd1d9282f (127.25.92.193:44453)
I20260805 23:10:41.520684 32643 heartbeater.cc:503] Master 127.25.92.254:39541 was elected leader, sending a full tablet report...
W20260805 23:10:41.564013 702 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:10:41.564301 702 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:10:41.564373 702 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:10:41.568584 702 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:10:41.568686 702 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:10:41.568778 702 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.92.194
I20260805 23:10:41.574498 702 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:44103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/krb5kdc/oryx_127.25.92.194.keytab
--principal=oryx/127.25.92.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.92.194:36939
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.92.194
--webserver_port=44929
--tserver_master_addrs=127.25.92.254:39541,127.25.92.253:38299,127.25.92.252:41853
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.702
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.92.194
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:10:41.575863 702 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:10:41.577098 702 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:10:41.587208 710 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:41.587285 707 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:41.587285 708 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:10:41.587554 702 server_base.cc:1034] running on GCE node
I20260805 23:10:41.588054 702 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:10:41.588693 702 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:10:41.589874 702 hybrid_clock.cc:648] HybridClock initialized: now 1785971441589844 us; error 51 us; skew 500 ppm
Aug 05 23:10:41 dist-test-slave-ss4n krb5kdc[32127](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971441, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.92.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:10:41.592778 702 init.cc:377] Logged in from keytab as oryx/127.25.92.194@KRBTEST.COM (short username oryx)
I20260805 23:10:41.593816 702 webserver.cc:460] Webserver started at http://127.25.92.194:44929/ using document root <none> and password file <none>
I20260805 23:10:41.594422 702 fs_manager.cc:359] Metadata directory not provided
I20260805 23:10:41.594506 702 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:10:41.598678 702 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260805 23:10:41.600939 717 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:41.602496 702 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:10:41.602649 702 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/ts-1/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/ts-1/wal
uuid: "fc9234f3fa864a598207c335d409c16c"
format_stamp: "Formatted at 2026-08-05 23:10:40 on dist-test-slave-ss4n"
server_key: "41c6ab6f38b82dfd39dc9c3693c764fb"
server_key_iv: "9c4ee8c76c788b06210409c75d443e95"
server_key_version: "encryptionkey@0"
I20260805 23:10:41.603258 702 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:10:41.644032 702 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:10:41.647420 702 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:10:41.647652 702 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:10:41.648286 702 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:10:41.649217 702 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:10:41.649294 702 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:41.649374 702 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:10:41.649431 702 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:41.659166 702 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.194:36939
I20260805 23:10:41.659189 833 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.194:36939 every 8 connection(s)
I20260805 23:10:41.660718 702 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/ts-1/data/info.pb
I20260805 23:10:41.669822 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 702
I20260805 23:10:41.670064 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 32510
Aug 05 23:10:41 dist-test-slave-ss4n krb5kdc[32127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971441, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.92.194@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM
Aug 05 23:10:41 dist-test-slave-ss4n krb5kdc[32127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971441, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.92.194@KRBTEST.COM for oryx/127.25.92.252@KRBTEST.COM
Aug 05 23:10:41 dist-test-slave-ss4n krb5kdc[32127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971441, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.92.194@KRBTEST.COM for oryx/127.25.92.253@KRBTEST.COM
I20260805 23:10:41.680785 698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:41.662976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.194:44207 (local address 127.25.92.252:41853)
0805 23:10:41.663138 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:41.663142 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:41.664418 (+ 1276us) server_negotiation.cc:407] Connection header received
0805 23:10:41.665740 (+ 1322us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:41.665744 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:41.665813 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:41.665917 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:41.667496 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:41.668301 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:41.669970 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:41.670394 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:41.670525 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:41.675094 (+ 4569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:41.675122 (+ 28us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:41.675125 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:41.675158 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:41.677440 (+ 2282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:41.678363 (+ 923us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:41.678367 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:41.678369 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:41.678495 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:41.679132 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:41.679136 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:41.679138 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:41.679446 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:41.679554 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:41.680424 (+ 870us) server_negotiation.cc:299] Negotiation successful
0805 23:10:41.680586 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260805 23:10:41.680785 449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:41.664750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.194:41395 (local address 127.25.92.253:38299)
0805 23:10:41.664882 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:41.664886 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:41.665080 (+ 194us) server_negotiation.cc:407] Connection header received
0805 23:10:41.666063 (+ 983us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:41.666067 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:41.666137 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:41.666229 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:41.667477 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:41.668309 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:41.669861 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:41.670226 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:41.670343 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:41.675854 (+ 5511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:41.675858 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:41.675861 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:41.675901 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:41.678442 (+ 2541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:41.679147 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:41.679150 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:41.679152 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:41.679261 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:41.679639 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:41.679643 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:41.679645 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:41.679906 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:41.680003 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:41.680424 (+ 421us) server_negotiation.cc:299] Negotiation successful
0805 23:10:41.680571 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260805 23:10:41.680871 32213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:41.663001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.194:51367 (local address 127.25.92.254:39541)
0805 23:10:41.663153 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:41.663158 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:41.664417 (+ 1259us) server_negotiation.cc:407] Connection header received
0805 23:10:41.665774 (+ 1357us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:41.665777 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:41.665846 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:41.665913 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:41.667463 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:41.668082 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:41.670896 (+ 2814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:41.671156 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:41.671283 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:41.674856 (+ 3573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:41.674865 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:41.674872 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:41.674916 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:41.678422 (+ 3506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:41.679205 (+ 783us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:41.679213 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:41.679218 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:41.679330 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:41.679690 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:41.679697 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:41.679701 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:41.679973 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:41.680078 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:41.680452 (+ 374us) server_negotiation.cc:299] Negotiation successful
0805 23:10:41.680616 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260805 23:10:41.682853 840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:41.664606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.253:38299 (local address 127.25.92.194:41395)
0805 23:10:41.664945 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:41.664959 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:41.665455 (+ 496us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:41.666279 (+ 824us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:41.666285 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:41.666735 (+ 450us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:41.667307 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:41.667315 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:41.668419 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:41.668422 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:41.669690 (+ 1268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:41.669702 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:41.670785 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:41.670793 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:41.671080 (+ 287us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:41.671915 (+ 835us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:10:41.671944 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:41.675682 (+ 3738us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:10:41.678581 (+ 2899us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:41.678588 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:41.678590 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:41.679012 (+ 422us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:41.679403 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:41.679406 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:41.679408 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:41.679520 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:41.680090 (+ 570us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:10:41.680093 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:10:41.680178 (+ 85us) client_negotiation.cc:769] Sending connection context
0805 23:10:41.680977 (+ 799us) client_negotiation.cc:240] Negotiation successful
0805 23:10:41.681555 (+ 578us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"mutex_wait_us":158,"spinlock_wait_cycles":16000,"thread_start_us":95,"threads_started":1}
I20260805 23:10:41.683176 839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:41.663431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:39541 (local address 127.25.92.194:51367)
0805 23:10:41.664242 (+ 811us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:41.664300 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:41.665538 (+ 1238us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:41.666160 (+ 622us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:41.666166 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:41.666717 (+ 551us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:41.667295 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:41.667310 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:41.668255 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:41.668262 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:41.670755 (+ 2493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:41.670765 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:41.671281 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:41.671285 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:41.671395 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:41.672096 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:10:41.672117 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:41.674561 (+ 2444us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:10:41.678591 (+ 4030us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:41.678597 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:41.678603 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:41.679062 (+ 459us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:41.679458 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:41.679463 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:41.679468 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:41.679580 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:41.680144 (+ 564us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:10:41.680150 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:10:41.680260 (+ 110us) client_negotiation.cc:769] Sending connection context
0805 23:10:41.681100 (+ 840us) client_negotiation.cc:240] Negotiation successful
0805 23:10:41.682195 (+ 1095us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":627,"mutex_wait_us":1856,"spinlock_wait_cycles":32512,"thread_start_us":165,"threads_started":1}
I20260805 23:10:41.682865 838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:41.663430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.252:41853 (local address 127.25.92.194:44207)
0805 23:10:41.664242 (+ 812us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:41.664300 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:41.665455 (+ 1155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:41.666136 (+ 681us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:41.666143 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:41.666728 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:41.667306 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:41.667315 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:41.668705 (+ 1390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:41.668709 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:41.669828 (+ 1119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:41.669843 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:41.671504 (+ 1661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:41.671511 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:41.671671 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:41.672310 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:10:41.672327 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:41.674889 (+ 2562us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:10:41.677696 (+ 2807us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:41.677712 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:41.677737 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:41.678220 (+ 483us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:41.678644 (+ 424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:41.678650 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:41.678654 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:41.678956 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:41.679689 (+ 733us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:10:41.679700 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:10:41.680090 (+ 390us) client_negotiation.cc:769] Sending connection context
0805 23:10:41.681691 (+ 1601us) client_negotiation.cc:240] Negotiation successful
0805 23:10:41.681835 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":604,"mutex_wait_us":86,"spinlock_wait_cycles":22656,"thread_start_us":161,"threads_started":1}
I20260805 23:10:41.684252 836 heartbeater.cc:347] Connected to a master server at 127.25.92.253:38299
I20260805 23:10:41.684325 834 heartbeater.cc:347] Connected to a master server at 127.25.92.252:41853
I20260805 23:10:41.684530 834 heartbeater.cc:464] Registering TS with master...
I20260805 23:10:41.684530 836 heartbeater.cc:464] Registering TS with master...
I20260805 23:10:41.684917 835 heartbeater.cc:347] Connected to a master server at 127.25.92.254:39541
I20260805 23:10:41.685006 835 heartbeater.cc:464] Registering TS with master...
I20260805 23:10:41.685252 835 heartbeater.cc:511] Master 127.25.92.254:39541 requested a full tablet report, sending...
I20260805 23:10:41.685254 836 heartbeater.cc:511] Master 127.25.92.253:38299 requested a full tablet report, sending...
I20260805 23:10:41.685254 834 heartbeater.cc:511] Master 127.25.92.252:41853 requested a full tablet report, sending...
I20260805 23:10:41.686545 32174 ts_manager.cc:194] Re-registered known tserver with Master: fc9234f3fa864a598207c335d409c16c (127.25.92.194:36939)
I20260805 23:10:41.686722 510 ts_manager.cc:194] Registered new tserver with Master: fc9234f3fa864a598207c335d409c16c (127.25.92.194:36939)
I20260805 23:10:41.686730 323 ts_manager.cc:194] Registered new tserver with Master: fc9234f3fa864a598207c335d409c16c (127.25.92.194:36939)
I20260805 23:10:41.687325 32174 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.25.92.194@KRBTEST.COM'} at 127.25.92.194:51367
I20260805 23:10:41.688597 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.92.195:41337
--local_ip_for_outbound_sockets=127.25.92.195
--tserver_master_addrs=127.25.92.254:39541
--webserver_port=43867
--webserver_interface=127.25.92.195
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/krb5kdc/oryx_127.25.92.195.keytab
--principal=oryx/127.25.92.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:44103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.25.92.254:39541,127.25.92.253:38299
--tserver_master_addrs=127.25.92.254:39541,127.25.92.253:38299,127.25.92.252:41853 with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:10:41.806461 841 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:10:41.806895 841 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:10:41.806978 841 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260805 23:10:41.809849 698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:41.806255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55612 (local address 127.25.92.252:41853)
0805 23:10:41.806400 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:41.806406 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:41.806499 (+ 93us) server_negotiation.cc:407] Connection header received
0805 23:10:41.806707 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:41.806711 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:41.806783 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:41.806895 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:10:41.807152 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:41.807685 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:41.808743 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:41.809198 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:41.809339 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:41.809554 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:41.809609 (+ 55us) server_negotiation.cc:299] Negotiation successful
0805 23:10:41.809695 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260805 23:10:41.809852 847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:41.806156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.252:41853 (local address 127.0.0.1:55612)
0805 23:10:41.806402 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:41.806424 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:41.806527 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:41.806912 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:41.806915 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:41.806937 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:10:41.807054 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:41.807060 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:41.807823 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:41.807826 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:41.808609 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:41.808618 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:41.809357 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:41.809360 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:41.809452 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:41.809501 (+ 49us) client_negotiation.cc:769] Sending connection context
0805 23:10:41.809592 (+ 91us) client_negotiation.cc:240] Negotiation successful
0805 23:10:41.809679 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":107,"threads_started":1}
I20260805 23:10:41.810649 521 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 4cc69a74b0534df2bbc6b22b987d8f63 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260805 23:10:41.811113 848 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4cc69a74b0534df2bbc6b22b987d8f63 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 87c2eb89255e45f0a2feae991859aa8e. Latest consensus state: current_term: 1 leader_uuid: "87c2eb89255e45f0a2feae991859aa8e" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "87c2eb89255e45f0a2feae991859aa8e" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 39541 } } peers { permanent_uuid: "d72d1f2dd8074bbbaf2fc4761feb7080" member_type: VOTER last_known_addr { host: "127.25.92.253" port: 38299 } attrs { promote: false } } peers { permanent_uuid: "4cc69a74b0534df2bbc6b22b987d8f63" member_type: NON_VOTER last_known_addr { host: "127.25.92.252" port: 41853 } attrs { promote: true } } }
I20260805 23:10:41.811271 848 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4cc69a74b0534df2bbc6b22b987d8f63 [sys.catalog]: This master's current role is: LEARNER
W20260805 23:10:41.812026 841 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:10:41.812131 841 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:10:41.812239 841 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.92.195
I20260805 23:10:41.818198 841 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:44103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/krb5kdc/oryx_127.25.92.195.keytab
--principal=oryx/127.25.92.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.92.195:41337
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.92.195
--webserver_port=43867
--tserver_master_addrs=127.25.92.254:39541,127.25.92.253:38299,127.25.92.252:41853
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.841
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.92.195
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:10:41.819355 841 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:10:41.820410 841 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:10:41.830389 851 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:41.830394 850 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:41.830394 853 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:10:41.830896 841 server_base.cc:1034] running on GCE node
I20260805 23:10:41.831321 841 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:10:41.831975 841 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:10:41.833138 841 hybrid_clock.cc:648] HybridClock initialized: now 1785971441833100 us; error 52 us; skew 500 ppm
Aug 05 23:10:41 dist-test-slave-ss4n krb5kdc[32127](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971441, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.92.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:10:41.836413 841 init.cc:377] Logged in from keytab as oryx/127.25.92.195@KRBTEST.COM (short username oryx)
I20260805 23:10:41.837605 841 webserver.cc:460] Webserver started at http://127.25.92.195:43867/ using document root <none> and password file <none>
I20260805 23:10:41.838191 841 fs_manager.cc:359] Metadata directory not provided
I20260805 23:10:41.838290 841 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:10:41.842396 841 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:10:41.844693 860 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:41.846264 841 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:10:41.846422 841 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/ts-2/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/ts-2/wal
uuid: "e6569ceea53644f6b26a1e4c960fe340"
format_stamp: "Formatted at 2026-08-05 23:10:40 on dist-test-slave-ss4n"
server_key: "c4228b7a91ec3c18f328a8988a47d8b3"
server_key_iv: "5b69eae7ec17440aeed7b1de5c34bce9"
server_key_version: "encryptionkey@0"
I20260805 23:10:41.847040 841 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:10:41.871660 841 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:10:41.875720 841 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:10:41.875970 841 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:10:41.876654 841 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:10:41.877696 841 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:10:41.877774 841 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:41.877878 841 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:10:41.877923 841 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:41.888932 841 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.195:41337
I20260805 23:10:41.889041 975 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.195:41337 every 8 connection(s)
I20260805 23:10:41.890081 841 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/ts-2/data/info.pb
I20260805 23:10:41.896090 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 841
Aug 05 23:10:41 dist-test-slave-ss4n krb5kdc[32127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971441, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.92.195@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM
Aug 05 23:10:41 dist-test-slave-ss4n krb5kdc[32127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971441, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.92.195@KRBTEST.COM for oryx/127.25.92.253@KRBTEST.COM
Aug 05 23:10:41 dist-test-slave-ss4n krb5kdc[32127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971441, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.92.195@KRBTEST.COM for oryx/127.25.92.252@KRBTEST.COM
I20260805 23:10:41.910980 698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:41.893728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.195:46869 (local address 127.25.92.252:41853)
0805 23:10:41.893866 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:41.893870 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:41.894047 (+ 177us) server_negotiation.cc:407] Connection header received
0805 23:10:41.894945 (+ 898us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:41.894949 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:41.895008 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:41.895060 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:41.896750 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:41.897599 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:41.901540 (+ 3941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:41.901932 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:41.902034 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:41.907276 (+ 5242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:41.907283 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:41.907289 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:41.907323 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:41.909048 (+ 1725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:41.909587 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:41.909590 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:41.909592 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:41.909672 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:41.909999 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:41.910003 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:41.910004 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:41.910205 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:41.910311 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:41.910699 (+ 388us) server_negotiation.cc:299] Negotiation successful
0805 23:10:41.910813 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260805 23:10:41.911454 32213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:41.892134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.195:50701 (local address 127.25.92.254:39541)
0805 23:10:41.892295 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:41.892300 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:41.893858 (+ 1558us) server_negotiation.cc:407] Connection header received
0805 23:10:41.894946 (+ 1088us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:41.894949 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:41.895005 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:41.895060 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:41.896719 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:41.897600 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:41.899516 (+ 1916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:41.899866 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:41.899973 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:41.904175 (+ 4202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:41.904180 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:41.904183 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:41.904214 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:41.908540 (+ 4326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:41.909023 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:41.909026 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:41.909028 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:41.909123 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:41.909428 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:41.909431 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:41.909433 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:41.909664 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:41.909753 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:41.911171 (+ 1418us) server_negotiation.cc:299] Negotiation successful
0805 23:10:41.911292 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260805 23:10:41.911798 982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:41.893609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.252:41853 (local address 127.25.92.195:46869)
0805 23:10:41.893945 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:41.893957 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:41.894738 (+ 781us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:41.895268 (+ 530us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:41.895275 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:41.895826 (+ 551us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:41.896528 (+ 702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:41.896550 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:41.897765 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:41.897768 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:41.901358 (+ 3590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:41.901370 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:41.904671 (+ 3301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:41.904676 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:41.904801 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:41.905468 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:10:41.905531 (+ 63us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:41.907106 (+ 1575us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:10:41.909202 (+ 2096us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:41.909205 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:41.909207 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:41.909454 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:41.909792 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:41.909794 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:41.909796 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:41.909900 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:41.910395 (+ 495us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:10:41.910401 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:10:41.910514 (+ 113us) client_negotiation.cc:769] Sending connection context
0805 23:10:41.910627 (+ 113us) client_negotiation.cc:240] Negotiation successful
0805 23:10:41.910766 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"mutex_wait_us":123,"spinlock_wait_cycles":29184,"thread_start_us":89,"threads_started":1}
I20260805 23:10:41.912170 980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:41.892549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:39541 (local address 127.25.92.195:50701)
0805 23:10:41.893410 (+ 861us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:41.893467 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:41.894738 (+ 1271us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:41.895268 (+ 530us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:41.895275 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:41.895826 (+ 551us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:41.896528 (+ 702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:41.896550 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:41.897765 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:41.897768 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:41.899379 (+ 1611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:41.899390 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:41.900139 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:41.900144 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:41.900305 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:41.901028 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:10:41.901057 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:41.903971 (+ 2914us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:10:41.908665 (+ 4694us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:41.908671 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:41.908676 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:41.908909 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:41.909229 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:41.909232 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:41.909234 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:41.909326 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:41.909832 (+ 506us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:10:41.909834 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:10:41.909916 (+ 82us) client_negotiation.cc:769] Sending connection context
0805 23:10:41.910057 (+ 141us) client_negotiation.cc:240] Negotiation successful
0805 23:10:41.910485 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":664,"mutex_wait_us":357,"spinlock_wait_cycles":122368,"thread_start_us":143,"threads_started":1}
I20260805 23:10:41.913235 449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:41.892044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.195:34009 (local address 127.25.92.253:38299)
0805 23:10:41.892217 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:41.892225 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:41.893858 (+ 1633us) server_negotiation.cc:407] Connection header received
0805 23:10:41.894931 (+ 1073us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:41.894935 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:41.895006 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:41.895086 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:41.896724 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:41.897559 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:41.899595 (+ 2036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:41.899836 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:41.899971 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:41.905079 (+ 5108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:41.905085 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:41.905090 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:41.905123 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:41.907015 (+ 1892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:41.908486 (+ 1471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:41.908491 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:41.908494 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:41.908608 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:41.909022 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:41.909026 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:41.909028 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:41.909269 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:41.909340 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:41.911411 (+ 2071us) server_negotiation.cc:299] Negotiation successful
0805 23:10:41.911529 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260805 23:10:41.913620 976 heartbeater.cc:347] Connected to a master server at 127.25.92.252:41853
I20260805 23:10:41.913877 976 heartbeater.cc:464] Registering TS with master...
I20260805 23:10:41.913914 977 heartbeater.cc:347] Connected to a master server at 127.25.92.254:39541
I20260805 23:10:41.913969 977 heartbeater.cc:464] Registering TS with master...
I20260805 23:10:41.914496 977 heartbeater.cc:511] Master 127.25.92.254:39541 requested a full tablet report, sending...
I20260805 23:10:41.914496 976 heartbeater.cc:511] Master 127.25.92.252:41853 requested a full tablet report, sending...
I20260805 23:10:41.915845 510 ts_manager.cc:194] Registered new tserver with Master: e6569ceea53644f6b26a1e4c960fe340 (127.25.92.195:41337)
I20260805 23:10:41.915992 32174 ts_manager.cc:194] Re-registered known tserver with Master: e6569ceea53644f6b26a1e4c960fe340 (127.25.92.195:41337)
I20260805 23:10:41.916882 32174 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.25.92.195@KRBTEST.COM'} at 127.25.92.195:50701
I20260805 23:10:41.919834 981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:41.892549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.253:38299 (local address 127.25.92.195:34009)
0805 23:10:41.893726 (+ 1177us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:41.893741 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:41.894714 (+ 973us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:41.895267 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:41.895274 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:41.895740 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:41.896531 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:41.896552 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:41.897760 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:41.897764 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:41.899386 (+ 1622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:41.899400 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:41.900012 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:41.900016 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:41.900228 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:41.901996 (+ 1768us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:10:41.902013 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:41.904405 (+ 2392us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:10:41.907936 (+ 3531us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:41.907945 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:41.907969 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:41.908354 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:41.908720 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:41.908725 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:41.908730 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:41.908912 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:41.909424 (+ 512us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:10:41.909431 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:10:41.909832 (+ 401us) client_negotiation.cc:769] Sending connection context
0805 23:10:41.918144 (+ 8312us) client_negotiation.cc:240] Negotiation successful
0805 23:10:41.919589 (+ 1445us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1079,"mutex_wait_us":1391,"spinlock_wait_cycles":133120,"thread_start_us":179,"threads_started":1}
I20260805 23:10:41.920790 987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:41.898311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40156 (local address 127.25.92.254:39541)
0805 23:10:41.899450 (+ 1139us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:41.899457 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:41.901053 (+ 1596us) server_negotiation.cc:407] Connection header received
0805 23:10:41.901235 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:41.901238 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:41.901287 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:41.901343 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:41.902460 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:41.903223 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:41.904363 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:41.904759 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:41.904847 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:41.906734 (+ 1887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:41.906741 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:41.906745 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:41.906798 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:41.913050 (+ 6252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:41.918592 (+ 5542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:41.918597 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:41.918599 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:41.918747 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:41.919553 (+ 806us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:41.919557 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:41.919559 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:41.919825 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:41.919986 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:41.920419 (+ 433us) server_negotiation.cc:299] Negotiation successful
0805 23:10:41.920583 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1049,"thread_start_us":78,"threads_started":1}
I20260805 23:10:41.920850 978 heartbeater.cc:347] Connected to a master server at 127.25.92.253:38299
I20260805 23:10:41.920964 978 heartbeater.cc:464] Registering TS with master...
I20260805 23:10:41.921205 978 heartbeater.cc:511] Master 127.25.92.253:38299 requested a full tablet report, sending...
I20260805 23:10:41.922406 323 ts_manager.cc:194] Registered new tserver with Master: e6569ceea53644f6b26a1e4c960fe340 (127.25.92.195:41337)
I20260805 23:10:41.923339 991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:41.899048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46586 (local address 127.25.92.252:41853)
0805 23:10:41.902384 (+ 3336us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:41.902388 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:41.905630 (+ 3242us) server_negotiation.cc:407] Connection header received
0805 23:10:41.906350 (+ 720us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:41.906354 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:41.906418 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:41.906487 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:41.907361 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:41.908188 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:41.909510 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:41.909871 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:41.909954 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:41.912209 (+ 2255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:41.912215 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:41.912221 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:41.912253 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:41.914686 (+ 2433us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:41.919170 (+ 4484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:41.919175 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:41.919177 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:41.919339 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:41.920636 (+ 1297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:41.920670 (+ 34us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:41.920676 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:41.921110 (+ 434us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:41.921283 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:41.922470 (+ 1187us) server_negotiation.cc:299] Negotiation successful
0805 23:10:41.922866 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3261,"thread_start_us":91,"threads_started":1}
I20260805 23:10:41.923524 988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:41.898527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57250 (local address 127.25.92.253:38299)
0805 23:10:41.899600 (+ 1073us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:41.899604 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:41.899616 (+ 12us) server_negotiation.cc:407] Connection header received
0805 23:10:41.899651 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:41.899653 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:41.899697 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:41.899776 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:41.900871 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:41.901659 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:41.902851 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:41.903182 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:41.903292 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:41.905167 (+ 1875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:41.905173 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:41.905178 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:41.905198 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:41.918747 (+ 13549us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:41.919542 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:41.919546 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:41.919550 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:41.919688 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:41.920211 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:41.920217 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:41.920221 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:41.921724 (+ 1503us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:41.921861 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:41.923108 (+ 1247us) server_negotiation.cc:299] Negotiation successful
0805 23:10:41.923315 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1001,"thread_start_us":79,"threads_started":1}
I20260805 23:10:41.925092 32174 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40156:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260805 23:10:41.928237 32174 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260805 23:10:41.931252 845 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87c2eb89255e45f0a2feae991859aa8e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "87c2eb89255e45f0a2feae991859aa8e" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "87c2eb89255e45f0a2feae991859aa8e" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 39541 } } peers { permanent_uuid: "d72d1f2dd8074bbbaf2fc4761feb7080" member_type: VOTER last_known_addr { host: "127.25.92.253" port: 38299 } attrs { promote: false } } peers { permanent_uuid: "4cc69a74b0534df2bbc6b22b987d8f63" member_type: NON_VOTER last_known_addr { host: "127.25.92.252" port: 41853 } attrs { promote: true } } }
I20260805 23:10:41.931442 845 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87c2eb89255e45f0a2feae991859aa8e [sys.catalog]: This master's current role is: LEADER
I20260805 23:10:41.936792 845 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 87c2eb89255e45f0a2feae991859aa8e: attempting to promote NON_VOTER 4cc69a74b0534df2bbc6b22b987d8f63 to VOTER
I20260805 23:10:41.937140 845 consensus_queue.cc:237] T 00000000000000000000000000000000 P 87c2eb89255e45f0a2feae991859aa8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "87c2eb89255e45f0a2feae991859aa8e" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 39541 } } peers { permanent_uuid: "d72d1f2dd8074bbbaf2fc4761feb7080" member_type: VOTER last_known_addr { host: "127.25.92.253" port: 38299 } attrs { promote: false } } peers { permanent_uuid: "4cc69a74b0534df2bbc6b22b987d8f63" member_type: VOTER last_known_addr { host: "127.25.92.252" port: 41853 } attrs { promote: false } }
I20260805 23:10:41.938539 333 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d72d1f2dd8074bbbaf2fc4761feb7080 [term 1 FOLLOWER]: Refusing update from remote peer 87c2eb89255e45f0a2feae991859aa8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260805 23:10:41.938537 521 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4cc69a74b0534df2bbc6b22b987d8f63 [term 1 LEARNER]: Refusing update from remote peer 87c2eb89255e45f0a2feae991859aa8e: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260805 23:10:41.938926 996 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 87c2eb89255e45f0a2feae991859aa8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "4cc69a74b0534df2bbc6b22b987d8f63" member_type: VOTER last_known_addr { host: "127.25.92.252" port: 41853 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260805 23:10:41.939091 846 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 87c2eb89255e45f0a2feae991859aa8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d72d1f2dd8074bbbaf2fc4761feb7080" member_type: VOTER last_known_addr { host: "127.25.92.253" port: 38299 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260805 23:10:41.941160 846 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 87c2eb89255e45f0a2feae991859aa8e [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 4cc69a74b0534df2bbc6b22b987d8f63 (127.25.92.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "87c2eb89255e45f0a2feae991859aa8e" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 39541 } } peers { permanent_uuid: "d72d1f2dd8074bbbaf2fc4761feb7080" member_type: VOTER last_known_addr { host: "127.25.92.253" port: 38299 } attrs { promote: false } } peers { permanent_uuid: "4cc69a74b0534df2bbc6b22b987d8f63" member_type: VOTER last_known_addr { host: "127.25.92.252" port: 41853 } attrs { promote: false } } }
I20260805 23:10:41.942955 845 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87c2eb89255e45f0a2feae991859aa8e [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "87c2eb89255e45f0a2feae991859aa8e" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "87c2eb89255e45f0a2feae991859aa8e" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 39541 } } peers { permanent_uuid: "d72d1f2dd8074bbbaf2fc4761feb7080" member_type: VOTER last_known_addr { host: "127.25.92.253" port: 38299 } attrs { promote: false } } peers { permanent_uuid: "4cc69a74b0534df2bbc6b22b987d8f63" member_type: VOTER last_known_addr { host: "127.25.92.252" port: 41853 } attrs { promote: false } } }
I20260805 23:10:41.943087 845 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87c2eb89255e45f0a2feae991859aa8e [sys.catalog]: This master's current role is: LEADER
I20260805 23:10:41.943035 846 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87c2eb89255e45f0a2feae991859aa8e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "87c2eb89255e45f0a2feae991859aa8e" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "87c2eb89255e45f0a2feae991859aa8e" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 39541 } } peers { permanent_uuid: "d72d1f2dd8074bbbaf2fc4761feb7080" member_type: VOTER last_known_addr { host: "127.25.92.253" port: 38299 } attrs { promote: false } } peers { permanent_uuid: "4cc69a74b0534df2bbc6b22b987d8f63" member_type: VOTER last_known_addr { host: "127.25.92.252" port: 41853 } attrs { promote: false } } }
I20260805 23:10:41.943159 846 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87c2eb89255e45f0a2feae991859aa8e [sys.catalog]: This master's current role is: LEADER
I20260805 23:10:41.943778 996 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87c2eb89255e45f0a2feae991859aa8e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "87c2eb89255e45f0a2feae991859aa8e" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "87c2eb89255e45f0a2feae991859aa8e" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 39541 } } peers { permanent_uuid: "d72d1f2dd8074bbbaf2fc4761feb7080" member_type: VOTER last_known_addr { host: "127.25.92.253" port: 38299 } attrs { promote: false } } peers { permanent_uuid: "4cc69a74b0534df2bbc6b22b987d8f63" member_type: VOTER last_known_addr { host: "127.25.92.252" port: 41853 } attrs { promote: false } } }
I20260805 23:10:41.943871 996 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87c2eb89255e45f0a2feae991859aa8e [sys.catalog]: This master's current role is: LEADER
I20260805 23:10:41.943751 333 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d72d1f2dd8074bbbaf2fc4761feb7080 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 4cc69a74b0534df2bbc6b22b987d8f63 (127.25.92.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "87c2eb89255e45f0a2feae991859aa8e" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 39541 } } peers { permanent_uuid: "d72d1f2dd8074bbbaf2fc4761feb7080" member_type: VOTER last_known_addr { host: "127.25.92.253" port: 38299 } attrs { promote: false } } peers { permanent_uuid: "4cc69a74b0534df2bbc6b22b987d8f63" member_type: VOTER last_known_addr { host: "127.25.92.252" port: 41853 } attrs { promote: false } } }
I20260805 23:10:41.944195 521 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 4cc69a74b0534df2bbc6b22b987d8f63 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 4cc69a74b0534df2bbc6b22b987d8f63 (127.25.92.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "87c2eb89255e45f0a2feae991859aa8e" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 39541 } } peers { permanent_uuid: "d72d1f2dd8074bbbaf2fc4761feb7080" member_type: VOTER last_known_addr { host: "127.25.92.253" port: 38299 } attrs { promote: false } } peers { permanent_uuid: "4cc69a74b0534df2bbc6b22b987d8f63" member_type: VOTER last_known_addr { host: "127.25.92.252" port: 41853 } attrs { promote: false } } }
I20260805 23:10:41.946586 848 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4cc69a74b0534df2bbc6b22b987d8f63 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "87c2eb89255e45f0a2feae991859aa8e" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "87c2eb89255e45f0a2feae991859aa8e" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 39541 } } peers { permanent_uuid: "d72d1f2dd8074bbbaf2fc4761feb7080" member_type: VOTER last_known_addr { host: "127.25.92.253" port: 38299 } attrs { promote: false } } peers { permanent_uuid: "4cc69a74b0534df2bbc6b22b987d8f63" member_type: VOTER last_known_addr { host: "127.25.92.252" port: 41853 } attrs { promote: false } } }
I20260805 23:10:41.946712 848 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4cc69a74b0534df2bbc6b22b987d8f63 [sys.catalog]: This master's current role is: FOLLOWER
I20260805 23:10:41.947520 1005 sys_catalog.cc:455] T 00000000000000000000000000000000 P d72d1f2dd8074bbbaf2fc4761feb7080 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "87c2eb89255e45f0a2feae991859aa8e" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "87c2eb89255e45f0a2feae991859aa8e" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 39541 } } peers { permanent_uuid: "d72d1f2dd8074bbbaf2fc4761feb7080" member_type: VOTER last_known_addr { host: "127.25.92.253" port: 38299 } attrs { promote: false } } peers { permanent_uuid: "4cc69a74b0534df2bbc6b22b987d8f63" member_type: VOTER last_known_addr { host: "127.25.92.252" port: 41853 } attrs { promote: false } } }
I20260805 23:10:41.947672 1005 sys_catalog.cc:458] T 00000000000000000000000000000000 P d72d1f2dd8074bbbaf2fc4761feb7080 [sys.catalog]: This master's current role is: FOLLOWER
I20260805 23:10:41.952396 847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:41.944845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.195:41337 (local address 127.0.0.1:35824)
0805 23:10:41.945782 (+ 937us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:41.945801 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:41.945895 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:41.948259 (+ 2364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:41.948262 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:41.948279 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:10:41.948388 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:41.948397 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:41.949677 (+ 1280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:41.949682 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:41.950622 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:41.950633 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:41.951822 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:41.951826 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:41.951994 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:41.952025 (+ 31us) client_negotiation.cc:769] Sending connection context
0805 23:10:41.952130 (+ 105us) client_negotiation.cc:240] Negotiation successful
0805 23:10:41.952242 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":822,"mutex_wait_us":62}
I20260805 23:10:41.952407 1009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:41.946529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35824 (local address 127.25.92.195:41337)
0805 23:10:41.947810 (+ 1281us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:41.947825 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:41.947850 (+ 25us) server_negotiation.cc:407] Connection header received
0805 23:10:41.947929 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:41.947933 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:41.948120 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:41.948293 (+ 173us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:10:41.948490 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:41.949559 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:41.950795 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:41.951700 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:41.951921 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:41.952074 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:41.952159 (+ 85us) server_negotiation.cc:299] Negotiation successful
0805 23:10:41.952250 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1175,"thread_start_us":264,"threads_started":1}
I20260805 23:10:41.953806 1006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:41.945160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.194:36939 (local address 127.0.0.1:60722)
0805 23:10:41.947183 (+ 2023us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:41.947201 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:41.947302 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:41.948663 (+ 1361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:41.948666 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:41.948684 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:10:41.948773 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:41.948779 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:41.950258 (+ 1479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:41.950262 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:41.951482 (+ 1220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:41.951573 (+ 91us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:41.953354 (+ 1781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:41.953358 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:41.953453 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:41.953527 (+ 74us) client_negotiation.cc:769] Sending connection context
0805 23:10:41.953602 (+ 75us) client_negotiation.cc:240] Negotiation successful
0805 23:10:41.953674 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1933,"thread_start_us":135,"threads_started":1}
I20260805 23:10:41.954312 1007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:41.945659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60722 (local address 127.25.92.194:36939)
0805 23:10:41.948293 (+ 2634us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:41.948302 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:41.948321 (+ 19us) server_negotiation.cc:407] Connection header received
0805 23:10:41.948384 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:41.948388 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:41.948535 (+ 147us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:41.948654 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:10:41.948912 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:41.950127 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:41.951708 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:41.952590 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:41.952729 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:41.952862 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:41.954076 (+ 1214us) server_negotiation.cc:299] Negotiation successful
0805 23:10:41.954182 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2531,"thread_start_us":99,"threads_started":1}
I20260805 23:10:41.955001 1004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:41.945648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.193:44453 (local address 127.0.0.1:35144)
0805 23:10:41.947911 (+ 2263us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:41.947926 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:41.948020 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:41.950308 (+ 2288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:41.950314 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:41.950339 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:10:41.950528 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:41.950536 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:41.951657 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:41.951663 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:41.953147 (+ 1484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:41.953157 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:41.954621 (+ 1464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:41.954625 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:41.954710 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:41.954735 (+ 25us) client_negotiation.cc:769] Sending connection context
0805 23:10:41.954795 (+ 60us) client_negotiation.cc:240] Negotiation successful
0805 23:10:41.954862 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2189,"mutex_wait_us":87,"thread_start_us":104,"threads_started":1}
I20260805 23:10:41.955207 1008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:41.946051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35144 (local address 127.25.92.193:44453)
0805 23:10:41.948947 (+ 2896us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:41.948966 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:41.948991 (+ 25us) server_negotiation.cc:407] Connection header received
0805 23:10:41.949094 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:41.949099 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:41.949290 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:41.950454 (+ 1164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:10:41.950640 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:41.951535 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:41.953288 (+ 1753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:41.954483 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:41.954615 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:41.954761 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:41.954835 (+ 74us) server_negotiation.cc:299] Negotiation successful
0805 23:10:41.954929 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2750,"thread_start_us":85,"threads_started":1}
I20260805 23:10:41.955758 909 tablet_service.cc:1467] Processing CreateTablet for tablet 91264e8a8c4b4377ada815ab2fd98aed (DEFAULT_TABLE table=test-table [id=c026bf17ee014a8295ae7cd9edc1c38e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:10:41.956954 909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 91264e8a8c4b4377ada815ab2fd98aed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:10:41.957237 766 tablet_service.cc:1467] Processing CreateTablet for tablet 91264e8a8c4b4377ada815ab2fd98aed (DEFAULT_TABLE table=test-table [id=c026bf17ee014a8295ae7cd9edc1c38e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:10:41.957787 627 tablet_service.cc:1467] Processing CreateTablet for tablet 91264e8a8c4b4377ada815ab2fd98aed (DEFAULT_TABLE table=test-table [id=c026bf17ee014a8295ae7cd9edc1c38e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:10:41.958273 766 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 91264e8a8c4b4377ada815ab2fd98aed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:10:41.958782 627 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 91264e8a8c4b4377ada815ab2fd98aed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:10:41.966862 1012 tablet_bootstrap.cc:492] T 91264e8a8c4b4377ada815ab2fd98aed P 64c076108dc34bc68cacff2bd1d9282f: Bootstrap starting.
I20260805 23:10:41.968899 1012 tablet_bootstrap.cc:654] T 91264e8a8c4b4377ada815ab2fd98aed P 64c076108dc34bc68cacff2bd1d9282f: Neither blocks nor log segments found. Creating new log.
I20260805 23:10:41.969741 1012 log.cc:824] T 91264e8a8c4b4377ada815ab2fd98aed P 64c076108dc34bc68cacff2bd1d9282f: Log is configured to *not* fsync() on all Append() calls
I20260805 23:10:41.969902 1011 tablet_bootstrap.cc:492] T 91264e8a8c4b4377ada815ab2fd98aed P fc9234f3fa864a598207c335d409c16c: Bootstrap starting.
I20260805 23:10:41.971765 1012 tablet_bootstrap.cc:492] T 91264e8a8c4b4377ada815ab2fd98aed P 64c076108dc34bc68cacff2bd1d9282f: No bootstrap required, opened a new log
I20260805 23:10:41.971979 1012 ts_tablet_manager.cc:1397] T 91264e8a8c4b4377ada815ab2fd98aed P 64c076108dc34bc68cacff2bd1d9282f: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260805 23:10:41.972329 1011 tablet_bootstrap.cc:654] T 91264e8a8c4b4377ada815ab2fd98aed P fc9234f3fa864a598207c335d409c16c: Neither blocks nor log segments found. Creating new log.
I20260805 23:10:41.973230 1011 log.cc:824] T 91264e8a8c4b4377ada815ab2fd98aed P fc9234f3fa864a598207c335d409c16c: Log is configured to *not* fsync() on all Append() calls
I20260805 23:10:41.974143 1010 tablet_bootstrap.cc:492] T 91264e8a8c4b4377ada815ab2fd98aed P e6569ceea53644f6b26a1e4c960fe340: Bootstrap starting.
I20260805 23:10:41.975984 1012 raft_consensus.cc:348] T 91264e8a8c4b4377ada815ab2fd98aed P 64c076108dc34bc68cacff2bd1d9282f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6569ceea53644f6b26a1e4c960fe340" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 41337 } } peers { permanent_uuid: "fc9234f3fa864a598207c335d409c16c" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 36939 } } peers { permanent_uuid: "64c076108dc34bc68cacff2bd1d9282f" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 44453 } }
I20260805 23:10:41.976320 1012 raft_consensus.cc:374] T 91264e8a8c4b4377ada815ab2fd98aed P 64c076108dc34bc68cacff2bd1d9282f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:10:41.976384 1012 raft_consensus.cc:729] T 91264e8a8c4b4377ada815ab2fd98aed P 64c076108dc34bc68cacff2bd1d9282f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64c076108dc34bc68cacff2bd1d9282f, State: Initialized, Role: FOLLOWER
I20260805 23:10:41.976790 1010 tablet_bootstrap.cc:654] T 91264e8a8c4b4377ada815ab2fd98aed P e6569ceea53644f6b26a1e4c960fe340: Neither blocks nor log segments found. Creating new log.
I20260805 23:10:41.976801 1012 consensus_queue.cc:260] T 91264e8a8c4b4377ada815ab2fd98aed P 64c076108dc34bc68cacff2bd1d9282f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6569ceea53644f6b26a1e4c960fe340" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 41337 } } peers { permanent_uuid: "fc9234f3fa864a598207c335d409c16c" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 36939 } } peers { permanent_uuid: "64c076108dc34bc68cacff2bd1d9282f" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 44453 } }
I20260805 23:10:41.977586 695 heartbeater.cc:503] Master 127.25.92.254:39541 was elected leader, sending a full tablet report...
I20260805 23:10:41.977787 1010 log.cc:824] T 91264e8a8c4b4377ada815ab2fd98aed P e6569ceea53644f6b26a1e4c960fe340: Log is configured to *not* fsync() on all Append() calls
I20260805 23:10:41.977825 1012 ts_tablet_manager.cc:1428] T 91264e8a8c4b4377ada815ab2fd98aed P 64c076108dc34bc68cacff2bd1d9282f: Time spent starting tablet: real 0.006s user 0.005s sys 0.002s
I20260805 23:10:41.980258 1011 tablet_bootstrap.cc:492] T 91264e8a8c4b4377ada815ab2fd98aed P fc9234f3fa864a598207c335d409c16c: No bootstrap required, opened a new log
I20260805 23:10:41.980566 1011 ts_tablet_manager.cc:1397] T 91264e8a8c4b4377ada815ab2fd98aed P fc9234f3fa864a598207c335d409c16c: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.000s
I20260805 23:10:41.984923 1011 raft_consensus.cc:348] T 91264e8a8c4b4377ada815ab2fd98aed P fc9234f3fa864a598207c335d409c16c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6569ceea53644f6b26a1e4c960fe340" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 41337 } } peers { permanent_uuid: "fc9234f3fa864a598207c335d409c16c" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 36939 } } peers { permanent_uuid: "64c076108dc34bc68cacff2bd1d9282f" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 44453 } }
I20260805 23:10:41.985170 1011 raft_consensus.cc:374] T 91264e8a8c4b4377ada815ab2fd98aed P fc9234f3fa864a598207c335d409c16c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:10:41.985208 1011 raft_consensus.cc:729] T 91264e8a8c4b4377ada815ab2fd98aed P fc9234f3fa864a598207c335d409c16c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc9234f3fa864a598207c335d409c16c, State: Initialized, Role: FOLLOWER
I20260805 23:10:41.985675 1011 consensus_queue.cc:260] T 91264e8a8c4b4377ada815ab2fd98aed P fc9234f3fa864a598207c335d409c16c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6569ceea53644f6b26a1e4c960fe340" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 41337 } } peers { permanent_uuid: "fc9234f3fa864a598207c335d409c16c" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 36939 } } peers { permanent_uuid: "64c076108dc34bc68cacff2bd1d9282f" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 44453 } }
I20260805 23:10:41.986814 1011 ts_tablet_manager.cc:1428] T 91264e8a8c4b4377ada815ab2fd98aed P fc9234f3fa864a598207c335d409c16c: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260805 23:10:41.987058 835 heartbeater.cc:503] Master 127.25.92.254:39541 was elected leader, sending a full tablet report...
I20260805 23:10:41.987605 1010 tablet_bootstrap.cc:492] T 91264e8a8c4b4377ada815ab2fd98aed P e6569ceea53644f6b26a1e4c960fe340: No bootstrap required, opened a new log
I20260805 23:10:41.987850 1010 ts_tablet_manager.cc:1397] T 91264e8a8c4b4377ada815ab2fd98aed P e6569ceea53644f6b26a1e4c960fe340: Time spent bootstrapping tablet: real 0.014s user 0.006s sys 0.000s
I20260805 23:10:41.992583 1010 raft_consensus.cc:348] T 91264e8a8c4b4377ada815ab2fd98aed P e6569ceea53644f6b26a1e4c960fe340 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6569ceea53644f6b26a1e4c960fe340" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 41337 } } peers { permanent_uuid: "fc9234f3fa864a598207c335d409c16c" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 36939 } } peers { permanent_uuid: "64c076108dc34bc68cacff2bd1d9282f" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 44453 } }
I20260805 23:10:41.992839 1010 raft_consensus.cc:374] T 91264e8a8c4b4377ada815ab2fd98aed P e6569ceea53644f6b26a1e4c960fe340 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:10:41.992877 1010 raft_consensus.cc:729] T 91264e8a8c4b4377ada815ab2fd98aed P e6569ceea53644f6b26a1e4c960fe340 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6569ceea53644f6b26a1e4c960fe340, State: Initialized, Role: FOLLOWER
I20260805 23:10:41.993366 1010 consensus_queue.cc:260] T 91264e8a8c4b4377ada815ab2fd98aed P e6569ceea53644f6b26a1e4c960fe340 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6569ceea53644f6b26a1e4c960fe340" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 41337 } } peers { permanent_uuid: "fc9234f3fa864a598207c335d409c16c" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 36939 } } peers { permanent_uuid: "64c076108dc34bc68cacff2bd1d9282f" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 44453 } }
I20260805 23:10:41.994215 977 heartbeater.cc:503] Master 127.25.92.254:39541 was elected leader, sending a full tablet report...
I20260805 23:10:41.994549 1010 ts_tablet_manager.cc:1428] T 91264e8a8c4b4377ada815ab2fd98aed P e6569ceea53644f6b26a1e4c960fe340: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260805 23:10:42.000473 1019 raft_consensus.cc:482] T 91264e8a8c4b4377ada815ab2fd98aed P e6569ceea53644f6b26a1e4c960fe340 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 23:10:42.000660 1019 raft_consensus.cc:504] T 91264e8a8c4b4377ada815ab2fd98aed P e6569ceea53644f6b26a1e4c960fe340 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6569ceea53644f6b26a1e4c960fe340" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 41337 } } peers { permanent_uuid: "fc9234f3fa864a598207c335d409c16c" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 36939 } } peers { permanent_uuid: "64c076108dc34bc68cacff2bd1d9282f" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 44453 } }
I20260805 23:10:42.001849 1019 leader_election.cc:288] T 91264e8a8c4b4377ada815ab2fd98aed P e6569ceea53644f6b26a1e4c960fe340 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc9234f3fa864a598207c335d409c16c (127.25.92.194:36939), 64c076108dc34bc68cacff2bd1d9282f (127.25.92.193:44453)
I20260805 23:10:42.006285 981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:42.002095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.194:36939 (local address 127.25.92.195:36575)
0805 23:10:42.002230 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:42.002247 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:42.002341 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:42.002684 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:42.002687 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:42.002704 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:10:42.002829 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:42.002835 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:42.003585 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:42.003589 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:42.004790 (+ 1201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:42.004801 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:42.005868 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:42.005871 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:42.005953 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:42.005972 (+ 19us) client_negotiation.cc:769] Sending connection context
0805 23:10:42.006037 (+ 65us) client_negotiation.cc:240] Negotiation successful
0805 23:10:42.006092 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260805 23:10:42.006313 1007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:42.002293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.195:36575 (local address 127.25.92.194:36939)
0805 23:10:42.002442 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:42.002446 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:42.002463 (+ 17us) server_negotiation.cc:407] Connection header received
0805 23:10:42.002516 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:42.002522 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:42.002579 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:42.002682 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:10:42.002949 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:42.003467 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:42.004954 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:42.005738 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:42.005877 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:42.005932 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:42.006075 (+ 143us) server_negotiation.cc:299] Negotiation successful
0805 23:10:42.006177 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260805 23:10:42.006614 980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:42.002099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.193:44453 (local address 127.25.92.195:35345)
0805 23:10:42.002592 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:42.002606 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:42.002725 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:42.003009 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:42.003012 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:42.003029 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:10:42.003144 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:42.003151 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:42.004139 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:42.004143 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:42.005133 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:42.005141 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:42.006202 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:42.006206 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:42.006312 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:42.006344 (+ 32us) client_negotiation.cc:769] Sending connection context
0805 23:10:42.006417 (+ 73us) client_negotiation.cc:240] Negotiation successful
0805 23:10:42.006514 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":376,"mutex_wait_us":316}
I20260805 23:10:42.007036 1008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:42.002232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.195:35345 (local address 127.25.92.193:44453)
0805 23:10:42.002382 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:42.002388 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:42.002681 (+ 293us) server_negotiation.cc:407] Connection header received
0805 23:10:42.002848 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:42.002851 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:42.002905 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:42.002987 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:10:42.003272 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:42.004005 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:42.005351 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:42.006076 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:42.006204 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:42.006254 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:42.006864 (+ 610us) server_negotiation.cc:299] Negotiation successful
0805 23:10:42.006948 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260805 23:10:42.007189 787 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91264e8a8c4b4377ada815ab2fd98aed" candidate_uuid: "e6569ceea53644f6b26a1e4c960fe340" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc9234f3fa864a598207c335d409c16c" is_pre_election: true
I20260805 23:10:42.007520 787 raft_consensus.cc:2454] T 91264e8a8c4b4377ada815ab2fd98aed P fc9234f3fa864a598207c335d409c16c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6569ceea53644f6b26a1e4c960fe340 in term 0.
I20260805 23:10:42.007588 647 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91264e8a8c4b4377ada815ab2fd98aed" candidate_uuid: "e6569ceea53644f6b26a1e4c960fe340" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64c076108dc34bc68cacff2bd1d9282f" is_pre_election: true
I20260805 23:10:42.007906 647 raft_consensus.cc:2454] T 91264e8a8c4b4377ada815ab2fd98aed P 64c076108dc34bc68cacff2bd1d9282f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6569ceea53644f6b26a1e4c960fe340 in term 0.
I20260805 23:10:42.008044 861 leader_election.cc:302] T 91264e8a8c4b4377ada815ab2fd98aed P e6569ceea53644f6b26a1e4c960fe340 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e6569ceea53644f6b26a1e4c960fe340, fc9234f3fa864a598207c335d409c16c; no voters:
I20260805 23:10:42.008383 1019 raft_consensus.cc:2781] T 91264e8a8c4b4377ada815ab2fd98aed P e6569ceea53644f6b26a1e4c960fe340 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260805 23:10:42.008481 1019 raft_consensus.cc:482] T 91264e8a8c4b4377ada815ab2fd98aed P e6569ceea53644f6b26a1e4c960fe340 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260805 23:10:42.008526 1019 raft_consensus.cc:3037] T 91264e8a8c4b4377ada815ab2fd98aed P e6569ceea53644f6b26a1e4c960fe340 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:10:42.009770 1019 raft_consensus.cc:504] T 91264e8a8c4b4377ada815ab2fd98aed P e6569ceea53644f6b26a1e4c960fe340 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6569ceea53644f6b26a1e4c960fe340" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 41337 } } peers { permanent_uuid: "fc9234f3fa864a598207c335d409c16c" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 36939 } } peers { permanent_uuid: "64c076108dc34bc68cacff2bd1d9282f" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 44453 } }
I20260805 23:10:42.010190 1019 leader_election.cc:288] T 91264e8a8c4b4377ada815ab2fd98aed P e6569ceea53644f6b26a1e4c960fe340 [CANDIDATE]: Term 1 election: Requested vote from peers fc9234f3fa864a598207c335d409c16c (127.25.92.194:36939), 64c076108dc34bc68cacff2bd1d9282f (127.25.92.193:44453)
I20260805 23:10:42.010560 787 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91264e8a8c4b4377ada815ab2fd98aed" candidate_uuid: "e6569ceea53644f6b26a1e4c960fe340" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc9234f3fa864a598207c335d409c16c"
I20260805 23:10:42.010566 647 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91264e8a8c4b4377ada815ab2fd98aed" candidate_uuid: "e6569ceea53644f6b26a1e4c960fe340" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64c076108dc34bc68cacff2bd1d9282f"
I20260805 23:10:42.010677 787 raft_consensus.cc:3037] T 91264e8a8c4b4377ada815ab2fd98aed P fc9234f3fa864a598207c335d409c16c [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:10:42.010746 647 raft_consensus.cc:3037] T 91264e8a8c4b4377ada815ab2fd98aed P 64c076108dc34bc68cacff2bd1d9282f [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:10:42.011762 787 raft_consensus.cc:2454] T 91264e8a8c4b4377ada815ab2fd98aed P fc9234f3fa864a598207c335d409c16c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6569ceea53644f6b26a1e4c960fe340 in term 1.
I20260805 23:10:42.012063 861 leader_election.cc:302] T 91264e8a8c4b4377ada815ab2fd98aed P e6569ceea53644f6b26a1e4c960fe340 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e6569ceea53644f6b26a1e4c960fe340, fc9234f3fa864a598207c335d409c16c; no voters:
I20260805 23:10:42.012095 647 raft_consensus.cc:2454] T 91264e8a8c4b4377ada815ab2fd98aed P 64c076108dc34bc68cacff2bd1d9282f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6569ceea53644f6b26a1e4c960fe340 in term 1.
I20260805 23:10:42.012223 1019 raft_consensus.cc:2781] T 91264e8a8c4b4377ada815ab2fd98aed P e6569ceea53644f6b26a1e4c960fe340 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:10:42.012449 1019 raft_consensus.cc:686] T 91264e8a8c4b4377ada815ab2fd98aed P e6569ceea53644f6b26a1e4c960fe340 [term 1 LEADER]: Becoming Leader. State: Replica: e6569ceea53644f6b26a1e4c960fe340, State: Running, Role: LEADER
I20260805 23:10:42.013008 1019 consensus_queue.cc:237] T 91264e8a8c4b4377ada815ab2fd98aed P e6569ceea53644f6b26a1e4c960fe340 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6569ceea53644f6b26a1e4c960fe340" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 41337 } } peers { permanent_uuid: "fc9234f3fa864a598207c335d409c16c" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 36939 } } peers { permanent_uuid: "64c076108dc34bc68cacff2bd1d9282f" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 44453 } }
I20260805 23:10:42.016754 32173 catalog_manager.cc:5495] T 91264e8a8c4b4377ada815ab2fd98aed P e6569ceea53644f6b26a1e4c960fe340 reported cstate change: term changed from 0 to 1, leader changed from <none> to e6569ceea53644f6b26a1e4c960fe340 (127.25.92.195). New cstate: current_term: 1 leader_uuid: "e6569ceea53644f6b26a1e4c960fe340" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6569ceea53644f6b26a1e4c960fe340" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 41337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc9234f3fa864a598207c335d409c16c" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 36939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64c076108dc34bc68cacff2bd1d9282f" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 44453 } health_report { overall_health: UNKNOWN } } }
I20260805 23:10:42.048241 1009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:42.044378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35838 (local address 127.25.92.195:41337)
0805 23:10:42.044554 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:42.044559 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:42.044574 (+ 15us) server_negotiation.cc:407] Connection header received
0805 23:10:42.044751 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:42.044754 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:42.044810 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:42.044927 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 23:10:42.045168 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:42.045778 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:42.046712 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:42.046938 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:42.047053 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:42.047368 (+ 315us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 23:10:42.047806 (+ 438us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 23:10:42.047945 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:42.048009 (+ 64us) server_negotiation.cc:299] Negotiation successful
0805 23:10:42.048067 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260805 23:10:42.054623 787 raft_consensus.cc:1261] T 91264e8a8c4b4377ada815ab2fd98aed P fc9234f3fa864a598207c335d409c16c [term 1 FOLLOWER]: Refusing update from remote peer e6569ceea53644f6b26a1e4c960fe340: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:10:42.054623 647 raft_consensus.cc:1261] T 91264e8a8c4b4377ada815ab2fd98aed P 64c076108dc34bc68cacff2bd1d9282f [term 1 FOLLOWER]: Refusing update from remote peer e6569ceea53644f6b26a1e4c960fe340: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:10:42.055510 1020 consensus_queue.cc:1035] T 91264e8a8c4b4377ada815ab2fd98aed P e6569ceea53644f6b26a1e4c960fe340 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64c076108dc34bc68cacff2bd1d9282f" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 44453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:10:42.055742 1019 consensus_queue.cc:1035] T 91264e8a8c4b4377ada815ab2fd98aed P e6569ceea53644f6b26a1e4c960fe340 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc9234f3fa864a598207c335d409c16c" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 36939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:10:42.067242 1025 mvcc.cc:205] Tried to move back new op lower bound from 7315339026647629824 to 7315339026488262656. Current Snapshot: MvccSnapshot[applied={T|T < 7315339026647629824}]
I20260805 23:10:42.074213 32174 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40156:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:10:42.074383 32174 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40156:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:10:42.079751 32174 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 87c2eb89255e45f0a2feae991859aa8e: Sending DeleteTablet for 3 replicas of tablet 91264e8a8c4b4377ada815ab2fd98aed
I20260805 23:10:42.080556 909 tablet_service.cc:1514] Processing DeleteTablet for tablet 91264e8a8c4b4377ada815ab2fd98aed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:10:42 UTC) from {username='oryx', principal='oryx/127.25.92.254@KRBTEST.COM'} at 127.0.0.1:35824
I20260805 23:10:42.080632 766 tablet_service.cc:1514] Processing DeleteTablet for tablet 91264e8a8c4b4377ada815ab2fd98aed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:10:42 UTC) from {username='oryx', principal='oryx/127.25.92.254@KRBTEST.COM'} at 127.0.0.1:60722
I20260805 23:10:42.081209 627 tablet_service.cc:1514] Processing DeleteTablet for tablet 91264e8a8c4b4377ada815ab2fd98aed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:10:42 UTC) from {username='oryx', principal='oryx/127.25.92.254@KRBTEST.COM'} at 127.0.0.1:35144
I20260805 23:10:42.081709 1034 tablet_replica.cc:330] T 91264e8a8c4b4377ada815ab2fd98aed P e6569ceea53644f6b26a1e4c960fe340: stopping tablet replica
I20260805 23:10:42.081784 1035 tablet_replica.cc:330] T 91264e8a8c4b4377ada815ab2fd98aed P 64c076108dc34bc68cacff2bd1d9282f: stopping tablet replica
I20260805 23:10:42.082142 1035 raft_consensus.cc:2229] T 91264e8a8c4b4377ada815ab2fd98aed P 64c076108dc34bc68cacff2bd1d9282f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 23:10:42.082134 1034 raft_consensus.cc:2229] T 91264e8a8c4b4377ada815ab2fd98aed P e6569ceea53644f6b26a1e4c960fe340 [term 1 LEADER]: Raft consensus shutting down.
I20260805 23:10:42.082585 1035 raft_consensus.cc:2258] T 91264e8a8c4b4377ada815ab2fd98aed P 64c076108dc34bc68cacff2bd1d9282f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:10:42.082612 1034 raft_consensus.cc:2258] T 91264e8a8c4b4377ada815ab2fd98aed P e6569ceea53644f6b26a1e4c960fe340 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:10:42.084251 1034 ts_tablet_manager.cc:1905] T 91264e8a8c4b4377ada815ab2fd98aed P e6569ceea53644f6b26a1e4c960fe340: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:10:42.084376 1035 ts_tablet_manager.cc:1905] T 91264e8a8c4b4377ada815ab2fd98aed P 64c076108dc34bc68cacff2bd1d9282f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:10:42.084578 1033 tablet_replica.cc:330] T 91264e8a8c4b4377ada815ab2fd98aed P fc9234f3fa864a598207c335d409c16c: stopping tablet replica
I20260805 23:10:42.084910 1033 raft_consensus.cc:2229] T 91264e8a8c4b4377ada815ab2fd98aed P fc9234f3fa864a598207c335d409c16c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 23:10:42.085170 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 32140
I20260805 23:10:42.085790 1033 raft_consensus.cc:2258] T 91264e8a8c4b4377ada815ab2fd98aed P fc9234f3fa864a598207c335d409c16c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:10:42.088402 1033 ts_tablet_manager.cc:1905] T 91264e8a8c4b4377ada815ab2fd98aed P fc9234f3fa864a598207c335d409c16c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:10:42.089533 1035 ts_tablet_manager.cc:1918] T 91264e8a8c4b4377ada815ab2fd98aed P 64c076108dc34bc68cacff2bd1d9282f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 23:10:42.089658 1035 log.cc:1196] T 91264e8a8c4b4377ada815ab2fd98aed P 64c076108dc34bc68cacff2bd1d9282f: Deleting WAL directory at /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/ts-0/wal/wals/91264e8a8c4b4377ada815ab2fd98aed
I20260805 23:10:42.090070 1035 ts_tablet_manager.cc:1939] T 91264e8a8c4b4377ada815ab2fd98aed P 64c076108dc34bc68cacff2bd1d9282f: Deleting consensus metadata
I20260805 23:10:42.092226 1033 ts_tablet_manager.cc:1918] T 91264e8a8c4b4377ada815ab2fd98aed P fc9234f3fa864a598207c335d409c16c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 23:10:42.092962 1034 ts_tablet_manager.cc:1918] T 91264e8a8c4b4377ada815ab2fd98aed P e6569ceea53644f6b26a1e4c960fe340: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 23:10:42.093102 1034 log.cc:1196] T 91264e8a8c4b4377ada815ab2fd98aed P e6569ceea53644f6b26a1e4c960fe340: Deleting WAL directory at /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/ts-2/wal/wals/91264e8a8c4b4377ada815ab2fd98aed
I20260805 23:10:42.093526 1034 ts_tablet_manager.cc:1939] T 91264e8a8c4b4377ada815ab2fd98aed P e6569ceea53644f6b26a1e4c960fe340: Deleting consensus metadata
I20260805 23:10:42.093616 1033 log.cc:1196] T 91264e8a8c4b4377ada815ab2fd98aed P fc9234f3fa864a598207c335d409c16c: Deleting WAL directory at /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/ts-1/wal/wals/91264e8a8c4b4377ada815ab2fd98aed
I20260805 23:10:42.094023 1033 ts_tablet_manager.cc:1939] T 91264e8a8c4b4377ada815ab2fd98aed P fc9234f3fa864a598207c335d409c16c: Deleting consensus metadata
W20260805 23:10:42.095396 722 connection.cc:535] server connection from 127.0.0.1:60722 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260805 23:10:42.095402 582 connection.cc:535] server connection from 127.0.0.1:35144 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260805 23:10:42.095414 861 connection.cc:535] server connection from 127.0.0.1:35824 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260805 23:10:42.108407 988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:42.083784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57254 (local address 127.25.92.253:38299)
0805 23:10:42.083898 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:42.083902 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:42.097005 (+ 13103us) server_negotiation.cc:407] Connection header received
0805 23:10:42.097286 (+ 281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:42.097290 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:42.097363 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:42.097428 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:42.098695 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:42.099522 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:42.100922 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:42.101314 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:42.101411 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:42.103892 (+ 2481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:42.103897 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:42.103901 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:42.103927 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:42.106503 (+ 2576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:42.106994 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:42.106998 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:42.107000 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:42.107085 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:42.107432 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:42.107435 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:42.107437 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:42.107654 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:42.107757 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:42.108098 (+ 341us) server_negotiation.cc:299] Negotiation successful
0805 23:10:42.108223 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260805 23:10:42.113132 449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:42.099712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57260 (local address 127.25.92.253:38299)
0805 23:10:42.099865 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:42.099871 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:42.100719 (+ 848us) server_negotiation.cc:407] Connection header received
0805 23:10:42.100912 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:42.100916 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:42.100966 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:42.101020 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:42.102865 (+ 1845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:42.103770 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:42.105124 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:42.105475 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:42.105620 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:42.107728 (+ 2108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:42.107735 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:42.107741 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:42.107779 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:42.109695 (+ 1916us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:42.110297 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:42.110300 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:42.110302 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:42.110393 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:42.111999 (+ 1606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:42.112004 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:42.112005 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:42.112255 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:42.112402 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:42.112808 (+ 406us) server_negotiation.cc:299] Negotiation successful
0805 23:10:42.112957 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260805 23:10:42.113672 698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:42.100274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46608 (local address 127.25.92.252:41853)
0805 23:10:42.100458 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:42.100463 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:42.100778 (+ 315us) server_negotiation.cc:407] Connection header received
0805 23:10:42.100959 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:42.100962 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:42.101010 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:42.101061 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:42.102218 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:42.102999 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:42.104127 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:42.104477 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:42.104547 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:42.107166 (+ 2619us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:42.107172 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:42.107178 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:42.107212 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:42.111115 (+ 3903us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:42.112004 (+ 889us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:42.112008 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:42.112010 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:42.112099 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:42.112537 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:42.112541 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:42.112543 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:42.112819 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:42.112931 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:42.113312 (+ 381us) server_negotiation.cc:299] Negotiation successful
0805 23:10:42.113454 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260805 23:10:42.118831 991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:42.083559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46596 (local address 127.25.92.252:41853)
0805 23:10:42.083689 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:42.083694 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:42.109395 (+ 25701us) server_negotiation.cc:407] Connection header received
0805 23:10:42.109648 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:42.109651 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:42.109701 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:42.109749 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:42.110547 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:42.111357 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:42.112512 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:42.112877 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:42.112947 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:42.115499 (+ 2552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:42.115503 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:42.115505 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:42.115526 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:42.116897 (+ 1371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:42.117394 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:42.117397 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:42.117398 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:42.117535 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:42.117896 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:42.117900 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:42.117902 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:42.118115 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:42.118219 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:42.118459 (+ 240us) server_negotiation.cc:299] Negotiation successful
0805 23:10:42.118626 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260805 23:10:43.072512 1046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:43.071669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:39541 (local address 127.25.92.193:53043)
0805 23:10:43.072261 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:43.072383 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:39541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":103,"threads_started":1}
W20260805 23:10:43.072885 695 heartbeater.cc:644] Failed to heartbeat to 127.25.92.254:39541 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.25.92.254:39541: connect: Connection refused (error 111)
I20260805 23:10:43.072955 1047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:43.072323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:39541 (local address 127.25.92.194:57989)
0805 23:10:43.072603 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:43.072857 (+ 254us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:39541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":68,"threads_started":1}
W20260805 23:10:43.073236 835 heartbeater.cc:644] Failed to heartbeat to 127.25.92.254:39541 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.25.92.254:39541: connect: Connection refused (error 111)
I20260805 23:10:43.073874 1048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:43.073492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:39541 (local address 127.25.92.195:60597)
0805 23:10:43.073721 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:43.073790 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:39541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":68,"threads_started":1}
W20260805 23:10:43.074162 977 heartbeater.cc:644] Failed to heartbeat to 127.25.92.254:39541 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.25.92.254:39541: connect: Connection refused (error 111)
I20260805 23:10:43.580250 1050 raft_consensus.cc:482] T 00000000000000000000000000000000 P d72d1f2dd8074bbbaf2fc4761feb7080 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 87c2eb89255e45f0a2feae991859aa8e)
I20260805 23:10:43.580267 1049 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4cc69a74b0534df2bbc6b22b987d8f63 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 87c2eb89255e45f0a2feae991859aa8e)
I20260805 23:10:43.580503 1050 raft_consensus.cc:504] T 00000000000000000000000000000000 P d72d1f2dd8074bbbaf2fc4761feb7080 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "87c2eb89255e45f0a2feae991859aa8e" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 39541 } } peers { permanent_uuid: "d72d1f2dd8074bbbaf2fc4761feb7080" member_type: VOTER last_known_addr { host: "127.25.92.253" port: 38299 } attrs { promote: false } } peers { permanent_uuid: "4cc69a74b0534df2bbc6b22b987d8f63" member_type: VOTER last_known_addr { host: "127.25.92.252" port: 41853 } attrs { promote: false } }
I20260805 23:10:43.580503 1049 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4cc69a74b0534df2bbc6b22b987d8f63 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "87c2eb89255e45f0a2feae991859aa8e" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 39541 } } peers { permanent_uuid: "d72d1f2dd8074bbbaf2fc4761feb7080" member_type: VOTER last_known_addr { host: "127.25.92.253" port: 38299 } attrs { promote: false } } peers { permanent_uuid: "4cc69a74b0534df2bbc6b22b987d8f63" member_type: VOTER last_known_addr { host: "127.25.92.252" port: 41853 } attrs { promote: false } }
I20260805 23:10:43.581707 1050 leader_election.cc:288] T 00000000000000000000000000000000 P d72d1f2dd8074bbbaf2fc4761feb7080 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 87c2eb89255e45f0a2feae991859aa8e (127.25.92.254:39541), 4cc69a74b0534df2bbc6b22b987d8f63 (127.25.92.252:41853)
I20260805 23:10:43.581703 1049 leader_election.cc:288] T 00000000000000000000000000000000 P 4cc69a74b0534df2bbc6b22b987d8f63 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 87c2eb89255e45f0a2feae991859aa8e (127.25.92.254:39541), d72d1f2dd8074bbbaf2fc4761feb7080 (127.25.92.253:38299)
I20260805 23:10:43.582417 1051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:43.581786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:39541 (local address 127.0.0.1:40606)
0805 23:10:43.582224 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:43.582309 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:39541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":70,"threads_started":1}
W20260805 23:10:43.582619 497 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.25.92.254:39541: connect: Connection refused (error 111)
I20260805 23:10:43.583017 1053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:43.582074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:39541 (local address 127.0.0.1:40612)
0805 23:10:43.582856 (+ 782us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:43.582929 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:39541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":703,"thread_start_us":80,"threads_started":1}
W20260805 23:10:43.583178 310 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.25.92.254:39541: connect: Connection refused (error 111)
I20260805 23:10:43.584100 1053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:43.583864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:39541 (local address 127.0.0.1:40630)
0805 23:10:43.583989 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:43.584040 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:39541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260805 23:10:43.584255 310 leader_election.cc:334] T 00000000000000000000000000000000 P d72d1f2dd8074bbbaf2fc4761feb7080 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 87c2eb89255e45f0a2feae991859aa8e (127.25.92.254:39541): Network error: Client connection negotiation failed: client connection to 127.25.92.254:39541: connect: Connection refused (error 111)
I20260805 23:10:43.584503 1055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:43.583839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:39541 (local address 127.0.0.1:40614)
0805 23:10:43.584360 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:43.584422 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:39541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":404}
W20260805 23:10:43.584633 497 leader_election.cc:334] T 00000000000000000000000000000000 P 4cc69a74b0534df2bbc6b22b987d8f63 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 87c2eb89255e45f0a2feae991859aa8e (127.25.92.254:39541): Network error: Client connection negotiation failed: client connection to 127.25.92.254:39541: connect: Connection refused (error 111)
I20260805 23:10:43.586798 1054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:43.582504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46620 (local address 127.25.92.252:41853)
0805 23:10:43.582916 (+ 412us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:43.582921 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:43.582936 (+ 15us) server_negotiation.cc:407] Connection header received
0805 23:10:43.582984 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:43.582988 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:43.583041 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:43.583490 (+ 449us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:10:43.583587 (+ 97us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:43.584604 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:43.585841 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:43.586250 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:43.586367 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:43.586426 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:43.586612 (+ 186us) server_negotiation.cc:299] Negotiation successful
0805 23:10:43.586686 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":56,"server-negotiator.queue_time_us":333,"spinlock_wait_cycles":3328,"thread_start_us":71,"threads_started":1}
I20260805 23:10:43.586792 1052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:43.582039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.252:41853 (local address 127.0.0.1:46620)
0805 23:10:43.582388 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:43.582402 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:43.582595 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:43.583213 (+ 618us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:43.583217 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:43.583236 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:10:43.583373 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:43.583382 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:43.584759 (+ 1377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:43.584766 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:43.585725 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:43.585738 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:43.586378 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:43.586382 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:43.586480 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:43.586504 (+ 24us) client_negotiation.cc:769] Sending connection context
0805 23:10:43.586598 (+ 94us) client_negotiation.cc:240] Negotiation successful
0805 23:10:43.586668 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":83,"threads_started":1}
I20260805 23:10:43.586987 1056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:43.582765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57264 (local address 127.25.92.253:38299)
0805 23:10:43.582998 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:43.583002 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:43.583230 (+ 228us) server_negotiation.cc:407] Connection header received
0805 23:10:43.583276 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:43.583279 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:43.583319 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:43.583407 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:10:43.583788 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:43.584623 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:43.585754 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:43.586314 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:43.586422 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:43.586489 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:43.586772 (+ 283us) server_negotiation.cc:299] Negotiation successful
0805 23:10:43.586847 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":64,"threads_started":1}
I20260805 23:10:43.586977 1051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:43.582602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.253:38299 (local address 127.0.0.1:57264)
0805 23:10:43.583043 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:43.583057 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:43.583149 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:43.583434 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:43.583439 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:43.583462 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:10:43.583644 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:43.583654 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:43.584758 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:43.584763 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:43.585636 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:43.585645 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:43.586558 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:43.586562 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:43.586638 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:43.586656 (+ 18us) client_negotiation.cc:769] Sending connection context
0805 23:10:43.586753 (+ 97us) client_negotiation.cc:240] Negotiation successful
0805 23:10:43.586830 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":387,"mutex_wait_us":25,"spinlock_wait_cycles":3456,"thread_start_us":90,"threads_started":1}
I20260805 23:10:43.587538 521 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d72d1f2dd8074bbbaf2fc4761feb7080" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "4cc69a74b0534df2bbc6b22b987d8f63" is_pre_election: true
I20260805 23:10:43.587707 333 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4cc69a74b0534df2bbc6b22b987d8f63" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "d72d1f2dd8074bbbaf2fc4761feb7080" is_pre_election: true
I20260805 23:10:43.587764 521 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4cc69a74b0534df2bbc6b22b987d8f63 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72d1f2dd8074bbbaf2fc4761feb7080 in term 1.
I20260805 23:10:43.587877 333 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d72d1f2dd8074bbbaf2fc4761feb7080 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4cc69a74b0534df2bbc6b22b987d8f63 in term 1.
I20260805 23:10:43.588148 307 leader_election.cc:302] T 00000000000000000000000000000000 P d72d1f2dd8074bbbaf2fc4761feb7080 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4cc69a74b0534df2bbc6b22b987d8f63, d72d1f2dd8074bbbaf2fc4761feb7080; no voters: 87c2eb89255e45f0a2feae991859aa8e
I20260805 23:10:43.588394 1050 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d72d1f2dd8074bbbaf2fc4761feb7080 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260805 23:10:43.588464 1050 raft_consensus.cc:482] T 00000000000000000000000000000000 P d72d1f2dd8074bbbaf2fc4761feb7080 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 87c2eb89255e45f0a2feae991859aa8e)
I20260805 23:10:43.588518 1050 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d72d1f2dd8074bbbaf2fc4761feb7080 [term 1 FOLLOWER]: Advancing to term 2
I20260805 23:10:43.588712 494 leader_election.cc:302] T 00000000000000000000000000000000 P 4cc69a74b0534df2bbc6b22b987d8f63 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4cc69a74b0534df2bbc6b22b987d8f63, d72d1f2dd8074bbbaf2fc4761feb7080; no voters: 87c2eb89255e45f0a2feae991859aa8e
I20260805 23:10:43.588940 1049 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4cc69a74b0534df2bbc6b22b987d8f63 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260805 23:10:43.589002 1049 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4cc69a74b0534df2bbc6b22b987d8f63 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 87c2eb89255e45f0a2feae991859aa8e)
I20260805 23:10:43.589054 1049 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4cc69a74b0534df2bbc6b22b987d8f63 [term 1 FOLLOWER]: Advancing to term 2
I20260805 23:10:43.589964 1050 raft_consensus.cc:504] T 00000000000000000000000000000000 P d72d1f2dd8074bbbaf2fc4761feb7080 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "87c2eb89255e45f0a2feae991859aa8e" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 39541 } } peers { permanent_uuid: "d72d1f2dd8074bbbaf2fc4761feb7080" member_type: VOTER last_known_addr { host: "127.25.92.253" port: 38299 } attrs { promote: false } } peers { permanent_uuid: "4cc69a74b0534df2bbc6b22b987d8f63" member_type: VOTER last_known_addr { host: "127.25.92.252" port: 41853 } attrs { promote: false } }
I20260805 23:10:43.590337 1049 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4cc69a74b0534df2bbc6b22b987d8f63 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "87c2eb89255e45f0a2feae991859aa8e" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 39541 } } peers { permanent_uuid: "d72d1f2dd8074bbbaf2fc4761feb7080" member_type: VOTER last_known_addr { host: "127.25.92.253" port: 38299 } attrs { promote: false } } peers { permanent_uuid: "4cc69a74b0534df2bbc6b22b987d8f63" member_type: VOTER last_known_addr { host: "127.25.92.252" port: 41853 } attrs { promote: false } }
I20260805 23:10:43.590415 1050 leader_election.cc:288] T 00000000000000000000000000000000 P d72d1f2dd8074bbbaf2fc4761feb7080 [CANDIDATE]: Term 2 election: Requested vote from peers 87c2eb89255e45f0a2feae991859aa8e (127.25.92.254:39541), 4cc69a74b0534df2bbc6b22b987d8f63 (127.25.92.252:41853)
I20260805 23:10:43.590709 1049 leader_election.cc:288] T 00000000000000000000000000000000 P 4cc69a74b0534df2bbc6b22b987d8f63 [CANDIDATE]: Term 2 election: Requested vote from peers 87c2eb89255e45f0a2feae991859aa8e (127.25.92.254:39541), d72d1f2dd8074bbbaf2fc4761feb7080 (127.25.92.253:38299)
I20260805 23:10:43.590721 521 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d72d1f2dd8074bbbaf2fc4761feb7080" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "4cc69a74b0534df2bbc6b22b987d8f63"
I20260805 23:10:43.590850 1052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:43.590548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:39541 (local address 127.0.0.1:40638)
0805 23:10:43.590709 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:43.590766 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:39541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260805 23:10:43.590893 521 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 4cc69a74b0534df2bbc6b22b987d8f63 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate d72d1f2dd8074bbbaf2fc4761feb7080 in current term 2: Already voted for candidate 4cc69a74b0534df2bbc6b22b987d8f63 in this term.
I20260805 23:10:43.591081 333 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4cc69a74b0534df2bbc6b22b987d8f63" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "d72d1f2dd8074bbbaf2fc4761feb7080"
I20260805 23:10:43.591243 333 raft_consensus.cc:2379] T 00000000000000000000000000000000 P d72d1f2dd8074bbbaf2fc4761feb7080 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 4cc69a74b0534df2bbc6b22b987d8f63 in current term 2: Already voted for candidate d72d1f2dd8074bbbaf2fc4761feb7080 in this term.
I20260805 23:10:43.591446 1051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:43.590941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:39541 (local address 127.0.0.1:40644)
0805 23:10:43.591306 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:43.591368 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:39541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":293}
I20260805 23:10:43.591562 1052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:43.591346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:39541 (local address 127.0.0.1:40658)
0805 23:10:43.591454 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:43.591500 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:39541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260805 23:10:43.591688 310 leader_election.cc:334] T 00000000000000000000000000000000 P d72d1f2dd8074bbbaf2fc4761feb7080 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 87c2eb89255e45f0a2feae991859aa8e (127.25.92.254:39541): Network error: Client connection negotiation failed: client connection to 127.25.92.254:39541: connect: Connection refused (error 111)
I20260805 23:10:43.591782 310 leader_election.cc:302] T 00000000000000000000000000000000 P d72d1f2dd8074bbbaf2fc4761feb7080 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d72d1f2dd8074bbbaf2fc4761feb7080; no voters: 4cc69a74b0534df2bbc6b22b987d8f63, 87c2eb89255e45f0a2feae991859aa8e
I20260805 23:10:43.591952 1050 raft_consensus.cc:2726] T 00000000000000000000000000000000 P d72d1f2dd8074bbbaf2fc4761feb7080 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260805 23:10:43.592228 1051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:43.591934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:39541 (local address 127.0.0.1:40674)
0805 23:10:43.592129 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:43.592176 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:39541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260805 23:10:43.592352 497 leader_election.cc:334] T 00000000000000000000000000000000 P 4cc69a74b0534df2bbc6b22b987d8f63 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 87c2eb89255e45f0a2feae991859aa8e (127.25.92.254:39541): Network error: Client connection negotiation failed: client connection to 127.25.92.254:39541: connect: Connection refused (error 111)
I20260805 23:10:43.592454 497 leader_election.cc:302] T 00000000000000000000000000000000 P 4cc69a74b0534df2bbc6b22b987d8f63 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4cc69a74b0534df2bbc6b22b987d8f63; no voters: 87c2eb89255e45f0a2feae991859aa8e, d72d1f2dd8074bbbaf2fc4761feb7080
I20260805 23:10:43.592571 1049 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 4cc69a74b0534df2bbc6b22b987d8f63 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260805 23:10:44.074059 1059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:44.073554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:39541 (local address 127.25.92.193:34303)
0805 23:10:44.073863 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:44.073936 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:39541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":96,"threads_started":1}
I20260805 23:10:44.074455 1060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:44.073845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:39541 (local address 127.25.92.194:55697)
0805 23:10:44.074279 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:44.074350 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:39541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":109,"threads_started":1}
I20260805 23:10:44.075138 1061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:44.074782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:39541 (local address 127.25.92.195:51229)
0805 23:10:44.075016 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:44.075065 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:39541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":83,"threads_started":1}
I20260805 23:10:45.075942 1062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:45.075088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:39541 (local address 127.25.92.193:56983)
0805 23:10:45.075763 (+ 675us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:45.075829 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:39541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":591,"thread_start_us":99,"threads_started":1}
I20260805 23:10:45.075980 1063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:45.075523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:39541 (local address 127.25.92.194:39401)
0805 23:10:45.075779 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:45.075846 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:39541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":80,"threads_started":1}
I20260805 23:10:45.076514 1065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:45.075797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:39541 (local address 127.25.92.195:35761)
0805 23:10:45.076291 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:45.076372 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:39541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":68,"threads_started":1}
I20260805 23:10:45.134827 1066 raft_consensus.cc:482] T 00000000000000000000000000000000 P d72d1f2dd8074bbbaf2fc4761feb7080 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 23:10:45.134969 1066 raft_consensus.cc:504] T 00000000000000000000000000000000 P d72d1f2dd8074bbbaf2fc4761feb7080 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "87c2eb89255e45f0a2feae991859aa8e" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 39541 } } peers { permanent_uuid: "d72d1f2dd8074bbbaf2fc4761feb7080" member_type: VOTER last_known_addr { host: "127.25.92.253" port: 38299 } attrs { promote: false } } peers { permanent_uuid: "4cc69a74b0534df2bbc6b22b987d8f63" member_type: VOTER last_known_addr { host: "127.25.92.252" port: 41853 } attrs { promote: false } }
I20260805 23:10:45.135437 1066 leader_election.cc:288] T 00000000000000000000000000000000 P d72d1f2dd8074bbbaf2fc4761feb7080 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 87c2eb89255e45f0a2feae991859aa8e (127.25.92.254:39541), 4cc69a74b0534df2bbc6b22b987d8f63 (127.25.92.252:41853)
I20260805 23:10:45.135979 521 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d72d1f2dd8074bbbaf2fc4761feb7080" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "4cc69a74b0534df2bbc6b22b987d8f63" is_pre_election: true
I20260805 23:10:45.136123 1067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:45.135632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:39541 (local address 127.0.0.1:40872)
0805 23:10:45.135949 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:45.136020 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:39541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":92,"threads_started":1}
I20260805 23:10:45.136164 521 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4cc69a74b0534df2bbc6b22b987d8f63 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72d1f2dd8074bbbaf2fc4761feb7080 in term 2.
I20260805 23:10:45.136562 307 leader_election.cc:302] T 00000000000000000000000000000000 P d72d1f2dd8074bbbaf2fc4761feb7080 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4cc69a74b0534df2bbc6b22b987d8f63, d72d1f2dd8074bbbaf2fc4761feb7080; no voters:
I20260805 23:10:45.136759 1066 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d72d1f2dd8074bbbaf2fc4761feb7080 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260805 23:10:45.136842 1066 raft_consensus.cc:482] T 00000000000000000000000000000000 P d72d1f2dd8074bbbaf2fc4761feb7080 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260805 23:10:45.136898 1066 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d72d1f2dd8074bbbaf2fc4761feb7080 [term 2 FOLLOWER]: Advancing to term 3
I20260805 23:10:45.137007 1067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:45.136735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:39541 (local address 127.0.0.1:40878)
0805 23:10:45.136855 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:45.136934 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:39541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260805 23:10:45.137126 310 leader_election.cc:334] T 00000000000000000000000000000000 P d72d1f2dd8074bbbaf2fc4761feb7080 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 87c2eb89255e45f0a2feae991859aa8e (127.25.92.254:39541): Network error: Client connection negotiation failed: client connection to 127.25.92.254:39541: connect: Connection refused (error 111)
I20260805 23:10:45.138202 1066 raft_consensus.cc:504] T 00000000000000000000000000000000 P d72d1f2dd8074bbbaf2fc4761feb7080 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "87c2eb89255e45f0a2feae991859aa8e" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 39541 } } peers { permanent_uuid: "d72d1f2dd8074bbbaf2fc4761feb7080" member_type: VOTER last_known_addr { host: "127.25.92.253" port: 38299 } attrs { promote: false } } peers { permanent_uuid: "4cc69a74b0534df2bbc6b22b987d8f63" member_type: VOTER last_known_addr { host: "127.25.92.252" port: 41853 } attrs { promote: false } }
I20260805 23:10:45.138569 1066 leader_election.cc:288] T 00000000000000000000000000000000 P d72d1f2dd8074bbbaf2fc4761feb7080 [CANDIDATE]: Term 3 election: Requested vote from peers 87c2eb89255e45f0a2feae991859aa8e (127.25.92.254:39541), 4cc69a74b0534df2bbc6b22b987d8f63 (127.25.92.252:41853)
I20260805 23:10:45.138960 1067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:45.138696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:39541 (local address 127.0.0.1:40882)
0805 23:10:45.138809 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:45.138862 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:39541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260805 23:10:45.138944 521 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d72d1f2dd8074bbbaf2fc4761feb7080" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "4cc69a74b0534df2bbc6b22b987d8f63"
I20260805 23:10:45.139115 521 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4cc69a74b0534df2bbc6b22b987d8f63 [term 2 FOLLOWER]: Advancing to term 3
I20260805 23:10:45.139626 1067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:45.139427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:39541 (local address 127.0.0.1:40896)
0805 23:10:45.139524 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:45.139569 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:39541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260805 23:10:45.139724 310 leader_election.cc:334] T 00000000000000000000000000000000 P d72d1f2dd8074bbbaf2fc4761feb7080 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 87c2eb89255e45f0a2feae991859aa8e (127.25.92.254:39541): Network error: Client connection negotiation failed: client connection to 127.25.92.254:39541: connect: Connection refused (error 111)
I20260805 23:10:45.140241 521 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4cc69a74b0534df2bbc6b22b987d8f63 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72d1f2dd8074bbbaf2fc4761feb7080 in term 3.
I20260805 23:10:45.140576 307 leader_election.cc:302] T 00000000000000000000000000000000 P d72d1f2dd8074bbbaf2fc4761feb7080 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4cc69a74b0534df2bbc6b22b987d8f63, d72d1f2dd8074bbbaf2fc4761feb7080; no voters: 87c2eb89255e45f0a2feae991859aa8e
I20260805 23:10:45.140758 1066 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d72d1f2dd8074bbbaf2fc4761feb7080 [term 3 FOLLOWER]: Leader election won for term 3
I20260805 23:10:45.140931 1066 raft_consensus.cc:686] T 00000000000000000000000000000000 P d72d1f2dd8074bbbaf2fc4761feb7080 [term 3 LEADER]: Becoming Leader. State: Replica: d72d1f2dd8074bbbaf2fc4761feb7080, State: Running, Role: LEADER
I20260805 23:10:45.141213 1066 consensus_queue.cc:237] T 00000000000000000000000000000000 P d72d1f2dd8074bbbaf2fc4761feb7080 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "87c2eb89255e45f0a2feae991859aa8e" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 39541 } } peers { permanent_uuid: "d72d1f2dd8074bbbaf2fc4761feb7080" member_type: VOTER last_known_addr { host: "127.25.92.253" port: 38299 } attrs { promote: false } } peers { permanent_uuid: "4cc69a74b0534df2bbc6b22b987d8f63" member_type: VOTER last_known_addr { host: "127.25.92.252" port: 41853 } attrs { promote: false } }
I20260805 23:10:45.142284 1069 sys_catalog.cc:455] T 00000000000000000000000000000000 P d72d1f2dd8074bbbaf2fc4761feb7080 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d72d1f2dd8074bbbaf2fc4761feb7080. Latest consensus state: current_term: 3 leader_uuid: "d72d1f2dd8074bbbaf2fc4761feb7080" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "87c2eb89255e45f0a2feae991859aa8e" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 39541 } } peers { permanent_uuid: "d72d1f2dd8074bbbaf2fc4761feb7080" member_type: VOTER last_known_addr { host: "127.25.92.253" port: 38299 } attrs { promote: false } } peers { permanent_uuid: "4cc69a74b0534df2bbc6b22b987d8f63" member_type: VOTER last_known_addr { host: "127.25.92.252" port: 41853 } attrs { promote: false } } }
I20260805 23:10:45.142419 1069 sys_catalog.cc:458] T 00000000000000000000000000000000 P d72d1f2dd8074bbbaf2fc4761feb7080 [sys.catalog]: This master's current role is: LEADER
I20260805 23:10:45.142738 1071 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:10:45.145143 1071 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:10:45.145603 1071 catalog_manager.cc:1259] Loaded cluster ID: cfa50e6d78c6414ca13b867ff0c3f044
I20260805 23:10:45.145650 1071 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:10:45.146210 1071 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:10:45.146588 1071 catalog_manager.cc:5879] T 00000000000000000000000000000000 P d72d1f2dd8074bbbaf2fc4761feb7080: Loaded TSK: 0
I20260805 23:10:45.147261 1071 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260805 23:10:45.184024 323 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57260:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260805 23:10:45.187248 323 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260805 23:10:45.190584 1067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:45.190326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:39541 (local address 127.0.0.1:40918)
0805 23:10:45.190469 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:45.190517 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:39541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260805 23:10:45.190558 521 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4cc69a74b0534df2bbc6b22b987d8f63 [term 3 FOLLOWER]: Refusing update from remote peer d72d1f2dd8074bbbaf2fc4761feb7080: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260805 23:10:45.191285 1067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:45.191049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:39541 (local address 127.0.0.1:40928)
0805 23:10:45.191157 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:45.191209 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:39541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260805 23:10:45.191200 1066 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d72d1f2dd8074bbbaf2fc4761feb7080 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4cc69a74b0534df2bbc6b22b987d8f63" member_type: VOTER last_known_addr { host: "127.25.92.252" port: 41853 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260805 23:10:45.191437 310 consensus_peers.cc:487] T 00000000000000000000000000000000 P d72d1f2dd8074bbbaf2fc4761feb7080 -> Peer 87c2eb89255e45f0a2feae991859aa8e (127.25.92.254:39541): Couldn't send request to peer 87c2eb89255e45f0a2feae991859aa8e. Status: Network error: Client connection negotiation failed: client connection to 127.25.92.254:39541: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260805 23:10:45.192683 1074 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4cc69a74b0534df2bbc6b22b987d8f63 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d72d1f2dd8074bbbaf2fc4761feb7080. Latest consensus state: current_term: 3 leader_uuid: "d72d1f2dd8074bbbaf2fc4761feb7080" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "87c2eb89255e45f0a2feae991859aa8e" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 39541 } } peers { permanent_uuid: "d72d1f2dd8074bbbaf2fc4761feb7080" member_type: VOTER last_known_addr { host: "127.25.92.253" port: 38299 } attrs { promote: false } } peers { permanent_uuid: "4cc69a74b0534df2bbc6b22b987d8f63" member_type: VOTER last_known_addr { host: "127.25.92.252" port: 41853 } attrs { promote: false } } }
I20260805 23:10:45.192803 1074 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4cc69a74b0534df2bbc6b22b987d8f63 [sys.catalog]: This master's current role is: FOLLOWER
I20260805 23:10:45.194027 1069 sys_catalog.cc:455] T 00000000000000000000000000000000 P d72d1f2dd8074bbbaf2fc4761feb7080 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "d72d1f2dd8074bbbaf2fc4761feb7080" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "87c2eb89255e45f0a2feae991859aa8e" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 39541 } } peers { permanent_uuid: "d72d1f2dd8074bbbaf2fc4761feb7080" member_type: VOTER last_known_addr { host: "127.25.92.253" port: 38299 } attrs { promote: false } } peers { permanent_uuid: "4cc69a74b0534df2bbc6b22b987d8f63" member_type: VOTER last_known_addr { host: "127.25.92.252" port: 41853 } attrs { promote: false } } }
I20260805 23:10:45.194177 1069 sys_catalog.cc:458] T 00000000000000000000000000000000 P d72d1f2dd8074bbbaf2fc4761feb7080 [sys.catalog]: This master's current role is: LEADER
I20260805 23:10:45.194576 1074 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4cc69a74b0534df2bbc6b22b987d8f63 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "d72d1f2dd8074bbbaf2fc4761feb7080" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "87c2eb89255e45f0a2feae991859aa8e" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 39541 } } peers { permanent_uuid: "d72d1f2dd8074bbbaf2fc4761feb7080" member_type: VOTER last_known_addr { host: "127.25.92.253" port: 38299 } attrs { promote: false } } peers { permanent_uuid: "4cc69a74b0534df2bbc6b22b987d8f63" member_type: VOTER last_known_addr { host: "127.25.92.252" port: 41853 } attrs { promote: false } } }
I20260805 23:10:45.194694 1074 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4cc69a74b0534df2bbc6b22b987d8f63 [sys.catalog]: This master's current role is: FOLLOWER
I20260805 23:10:45.205072 1081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:45.200247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.193:44453 (local address 127.0.0.1:35156)
0805 23:10:45.200508 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:45.200524 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:45.200623 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:45.201272 (+ 649us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:45.201276 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:45.201303 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:10:45.201516 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:45.201527 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:45.202531 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:45.202537 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:45.203702 (+ 1165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:45.203712 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:45.204614 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:45.204618 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:45.204720 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:45.204745 (+ 25us) client_negotiation.cc:769] Sending connection context
0805 23:10:45.204829 (+ 84us) client_negotiation.cc:240] Negotiation successful
0805 23:10:45.204906 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"mutex_wait_us":45,"spinlock_wait_cycles":896,"thread_start_us":93,"threads_started":1}
I20260805 23:10:45.205129 1083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:45.200591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35156 (local address 127.25.92.193:44453)
0805 23:10:45.201018 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:45.201023 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:45.201038 (+ 15us) server_negotiation.cc:407] Connection header received
0805 23:10:45.201095 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:45.201097 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:45.201163 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:45.201246 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:10:45.201685 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:45.202425 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:45.203840 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:45.204509 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:45.204616 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:45.204684 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:45.204859 (+ 175us) server_negotiation.cc:299] Negotiation successful
0805 23:10:45.204956 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":91,"threads_started":1}
I20260805 23:10:45.205241 1067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:45.199932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.195:41337 (local address 127.0.0.1:35854)
0805 23:10:45.200044 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:45.200059 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:45.200159 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:45.201281 (+ 1122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:45.201283 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:45.201308 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:10:45.201726 (+ 418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:45.201732 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:45.202900 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:45.202903 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:45.203979 (+ 1076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:45.203992 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:45.204879 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:45.204882 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:45.204968 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:45.204989 (+ 21us) client_negotiation.cc:769] Sending connection context
0805 23:10:45.205034 (+ 45us) client_negotiation.cc:240] Negotiation successful
0805 23:10:45.205109 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":245,"spinlock_wait_cycles":5248}
I20260805 23:10:45.205696 1082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:45.200571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.194:36939 (local address 127.0.0.1:60728)
0805 23:10:45.201440 (+ 869us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:45.201453 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:45.201569 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:45.201949 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:45.201952 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:45.201970 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:10:45.202042 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:45.202047 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:45.203101 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:45.203105 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:45.204079 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:45.204090 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:45.205190 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:45.205194 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:45.205307 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:45.205338 (+ 31us) client_negotiation.cc:769] Sending connection context
0805 23:10:45.205426 (+ 88us) client_negotiation.cc:240] Negotiation successful
0805 23:10:45.205548 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":815,"mutex_wait_us":1,"thread_start_us":96,"threads_started":1}
I20260805 23:10:45.205724 1080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:45.200099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35854 (local address 127.25.92.195:41337)
0805 23:10:45.200995 (+ 896us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:45.201000 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:45.201018 (+ 18us) server_negotiation.cc:407] Connection header received
0805 23:10:45.201081 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:45.201084 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:45.201143 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:45.201402 (+ 259us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:10:45.201860 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:45.202774 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:45.204120 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:45.204755 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:45.205401 (+ 646us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:45.205458 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:45.205519 (+ 61us) server_negotiation.cc:299] Negotiation successful
0805 23:10:45.205589 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":802,"thread_start_us":114,"threads_started":1}
I20260805 23:10:45.205729 1084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:45.200951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60728 (local address 127.25.92.194:36939)
0805 23:10:45.201350 (+ 399us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:45.201355 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:45.201664 (+ 309us) server_negotiation.cc:407] Connection header received
0805 23:10:45.201734 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:45.201736 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:45.201824 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:45.201942 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:10:45.202131 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:45.202967 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:45.204230 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:45.204948 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:45.205057 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:45.205119 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:45.205460 (+ 341us) server_negotiation.cc:299] Negotiation successful
0805 23:10:45.205583 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":93,"threads_started":1}
I20260805 23:10:45.206280 909 tablet_service.cc:1467] Processing CreateTablet for tablet 4f127de797db42249900d2dbb9e072bf (DEFAULT_TABLE table=test-table [id=a9bd7ac2837a4d48a8ba8ea6c0ab4060]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:10:45.206471 766 tablet_service.cc:1467] Processing CreateTablet for tablet 4f127de797db42249900d2dbb9e072bf (DEFAULT_TABLE table=test-table [id=a9bd7ac2837a4d48a8ba8ea6c0ab4060]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:10:45.206637 909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f127de797db42249900d2dbb9e072bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:10:45.206615 627 tablet_service.cc:1467] Processing CreateTablet for tablet 4f127de797db42249900d2dbb9e072bf (DEFAULT_TABLE table=test-table [id=a9bd7ac2837a4d48a8ba8ea6c0ab4060]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:10:45.206784 766 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f127de797db42249900d2dbb9e072bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:10:45.206885 627 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f127de797db42249900d2dbb9e072bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:10:45.210799 1087 tablet_bootstrap.cc:492] T 4f127de797db42249900d2dbb9e072bf P e6569ceea53644f6b26a1e4c960fe340: Bootstrap starting.
I20260805 23:10:45.211030 1086 tablet_bootstrap.cc:492] T 4f127de797db42249900d2dbb9e072bf P 64c076108dc34bc68cacff2bd1d9282f: Bootstrap starting.
I20260805 23:10:45.211939 1086 tablet_bootstrap.cc:654] T 4f127de797db42249900d2dbb9e072bf P 64c076108dc34bc68cacff2bd1d9282f: Neither blocks nor log segments found. Creating new log.
I20260805 23:10:45.212028 1087 tablet_bootstrap.cc:654] T 4f127de797db42249900d2dbb9e072bf P e6569ceea53644f6b26a1e4c960fe340: Neither blocks nor log segments found. Creating new log.
I20260805 23:10:45.212671 1085 tablet_bootstrap.cc:492] T 4f127de797db42249900d2dbb9e072bf P fc9234f3fa864a598207c335d409c16c: Bootstrap starting.
I20260805 23:10:45.213305 1086 tablet_bootstrap.cc:492] T 4f127de797db42249900d2dbb9e072bf P 64c076108dc34bc68cacff2bd1d9282f: No bootstrap required, opened a new log
I20260805 23:10:45.213305 1087 tablet_bootstrap.cc:492] T 4f127de797db42249900d2dbb9e072bf P e6569ceea53644f6b26a1e4c960fe340: No bootstrap required, opened a new log
I20260805 23:10:45.213413 1086 ts_tablet_manager.cc:1397] T 4f127de797db42249900d2dbb9e072bf P 64c076108dc34bc68cacff2bd1d9282f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260805 23:10:45.213435 1085 tablet_bootstrap.cc:654] T 4f127de797db42249900d2dbb9e072bf P fc9234f3fa864a598207c335d409c16c: Neither blocks nor log segments found. Creating new log.
I20260805 23:10:45.213599 1087 ts_tablet_manager.cc:1397] T 4f127de797db42249900d2dbb9e072bf P e6569ceea53644f6b26a1e4c960fe340: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260805 23:10:45.213958 1086 raft_consensus.cc:348] T 4f127de797db42249900d2dbb9e072bf P 64c076108dc34bc68cacff2bd1d9282f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6569ceea53644f6b26a1e4c960fe340" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 41337 } } peers { permanent_uuid: "64c076108dc34bc68cacff2bd1d9282f" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 44453 } } peers { permanent_uuid: "fc9234f3fa864a598207c335d409c16c" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 36939 } }
I20260805 23:10:45.214115 1086 raft_consensus.cc:374] T 4f127de797db42249900d2dbb9e072bf P 64c076108dc34bc68cacff2bd1d9282f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:10:45.214160 1086 raft_consensus.cc:729] T 4f127de797db42249900d2dbb9e072bf P 64c076108dc34bc68cacff2bd1d9282f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64c076108dc34bc68cacff2bd1d9282f, State: Initialized, Role: FOLLOWER
I20260805 23:10:45.214073 1087 raft_consensus.cc:348] T 4f127de797db42249900d2dbb9e072bf P e6569ceea53644f6b26a1e4c960fe340 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6569ceea53644f6b26a1e4c960fe340" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 41337 } } peers { permanent_uuid: "64c076108dc34bc68cacff2bd1d9282f" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 44453 } } peers { permanent_uuid: "fc9234f3fa864a598207c335d409c16c" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 36939 } }
I20260805 23:10:45.214344 1086 consensus_queue.cc:260] T 4f127de797db42249900d2dbb9e072bf P 64c076108dc34bc68cacff2bd1d9282f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6569ceea53644f6b26a1e4c960fe340" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 41337 } } peers { permanent_uuid: "64c076108dc34bc68cacff2bd1d9282f" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 44453 } } peers { permanent_uuid: "fc9234f3fa864a598207c335d409c16c" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 36939 } }
I20260805 23:10:45.214560 1087 raft_consensus.cc:374] T 4f127de797db42249900d2dbb9e072bf P e6569ceea53644f6b26a1e4c960fe340 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:10:45.214656 1087 raft_consensus.cc:729] T 4f127de797db42249900d2dbb9e072bf P e6569ceea53644f6b26a1e4c960fe340 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6569ceea53644f6b26a1e4c960fe340, State: Initialized, Role: FOLLOWER
I20260805 23:10:45.214774 1086 ts_tablet_manager.cc:1428] T 4f127de797db42249900d2dbb9e072bf P 64c076108dc34bc68cacff2bd1d9282f: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260805 23:10:45.214916 1087 consensus_queue.cc:260] T 4f127de797db42249900d2dbb9e072bf P e6569ceea53644f6b26a1e4c960fe340 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6569ceea53644f6b26a1e4c960fe340" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 41337 } } peers { permanent_uuid: "64c076108dc34bc68cacff2bd1d9282f" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 44453 } } peers { permanent_uuid: "fc9234f3fa864a598207c335d409c16c" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 36939 } }
I20260805 23:10:45.215526 1062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:45.215256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:39541 (local address 127.25.92.193:46035)
0805 23:10:45.215367 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:45.215433 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:39541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260805 23:10:45.216249 1062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:45.216038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:39541 (local address 127.25.92.193:44987)
0805 23:10:45.216134 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:45.216189 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:39541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260805 23:10:45.216289 1087 ts_tablet_manager.cc:1428] T 4f127de797db42249900d2dbb9e072bf P e6569ceea53644f6b26a1e4c960fe340: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260805 23:10:45.217305 1065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:45.216994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:39541 (local address 127.25.92.195:42407)
0805 23:10:45.217173 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:45.217234 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:39541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260805 23:10:45.217490 696 heartbeater.cc:503] Master 127.25.92.253:38299 was elected leader, sending a full tablet report...
I20260805 23:10:45.218822 1065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:45.218560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:39541 (local address 127.25.92.195:47845)
0805 23:10:45.218685 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:45.218763 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:39541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260805 23:10:45.219285 978 heartbeater.cc:503] Master 127.25.92.253:38299 was elected leader, sending a full tablet report...
I20260805 23:10:45.219296 1085 tablet_bootstrap.cc:492] T 4f127de797db42249900d2dbb9e072bf P fc9234f3fa864a598207c335d409c16c: No bootstrap required, opened a new log
I20260805 23:10:45.219411 1085 ts_tablet_manager.cc:1397] T 4f127de797db42249900d2dbb9e072bf P fc9234f3fa864a598207c335d409c16c: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260805 23:10:45.220001 1085 raft_consensus.cc:348] T 4f127de797db42249900d2dbb9e072bf P fc9234f3fa864a598207c335d409c16c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6569ceea53644f6b26a1e4c960fe340" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 41337 } } peers { permanent_uuid: "64c076108dc34bc68cacff2bd1d9282f" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 44453 } } peers { permanent_uuid: "fc9234f3fa864a598207c335d409c16c" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 36939 } }
I20260805 23:10:45.220139 1085 raft_consensus.cc:374] T 4f127de797db42249900d2dbb9e072bf P fc9234f3fa864a598207c335d409c16c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:10:45.220178 1085 raft_consensus.cc:729] T 4f127de797db42249900d2dbb9e072bf P fc9234f3fa864a598207c335d409c16c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc9234f3fa864a598207c335d409c16c, State: Initialized, Role: FOLLOWER
I20260805 23:10:45.220351 1085 consensus_queue.cc:260] T 4f127de797db42249900d2dbb9e072bf P fc9234f3fa864a598207c335d409c16c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6569ceea53644f6b26a1e4c960fe340" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 41337 } } peers { permanent_uuid: "64c076108dc34bc68cacff2bd1d9282f" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 44453 } } peers { permanent_uuid: "fc9234f3fa864a598207c335d409c16c" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 36939 } }
I20260805 23:10:45.220819 1085 ts_tablet_manager.cc:1428] T 4f127de797db42249900d2dbb9e072bf P fc9234f3fa864a598207c335d409c16c: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260805 23:10:45.221451 1063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:45.221165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:39541 (local address 127.25.92.194:58823)
0805 23:10:45.221327 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:45.221381 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:39541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260805 23:10:45.222409 1063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:45.222186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:39541 (local address 127.25.92.194:49597)
0805 23:10:45.222299 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:45.222346 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:39541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260805 23:10:45.222565 836 heartbeater.cc:503] Master 127.25.92.253:38299 was elected leader, sending a full tablet report...
I20260805 23:10:45.240768 1093 raft_consensus.cc:482] T 4f127de797db42249900d2dbb9e072bf P fc9234f3fa864a598207c335d409c16c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 23:10:45.240945 1093 raft_consensus.cc:504] T 4f127de797db42249900d2dbb9e072bf P fc9234f3fa864a598207c335d409c16c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6569ceea53644f6b26a1e4c960fe340" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 41337 } } peers { permanent_uuid: "64c076108dc34bc68cacff2bd1d9282f" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 44453 } } peers { permanent_uuid: "fc9234f3fa864a598207c335d409c16c" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 36939 } }
I20260805 23:10:45.242285 1093 leader_election.cc:288] T 4f127de797db42249900d2dbb9e072bf P fc9234f3fa864a598207c335d409c16c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e6569ceea53644f6b26a1e4c960fe340 (127.25.92.195:41337), 64c076108dc34bc68cacff2bd1d9282f (127.25.92.193:44453)
I20260805 23:10:45.246158 1080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:45.242582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.194:41525 (local address 127.25.92.195:41337)
0805 23:10:45.242723 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:45.242729 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:45.242744 (+ 15us) server_negotiation.cc:407] Connection header received
0805 23:10:45.242802 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:45.242804 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:45.242844 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:45.242948 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:10:45.243353 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:45.243872 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:45.244941 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:45.245504 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:45.245624 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:45.245687 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:45.245961 (+ 274us) server_negotiation.cc:299] Negotiation successful
0805 23:10:45.246039 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260805 23:10:45.246187 1063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:45.242483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.195:41337 (local address 127.25.92.194:41525)
0805 23:10:45.242590 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:45.242608 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:45.242719 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:45.243038 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:45.243040 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:45.243063 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:10:45.243224 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:45.243233 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:45.243988 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:45.243993 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:45.244811 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:45.244819 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:45.245746 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:45.245749 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:45.245830 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:45.245851 (+ 21us) client_negotiation.cc:769] Sending connection context
0805 23:10:45.245943 (+ 92us) client_negotiation.cc:240] Negotiation successful
0805 23:10:45.246021 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28,"mutex_wait_us":1}
I20260805 23:10:45.246353 1094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:45.242738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.193:44453 (local address 127.25.92.194:56903)
0805 23:10:45.243051 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:45.243063 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:45.243139 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:45.243364 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:45.243367 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:45.243387 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:10:45.243484 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:45.243489 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:45.244231 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:45.244237 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:45.245078 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:45.245092 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:45.245943 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:45.245946 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:45.246038 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:45.246062 (+ 24us) client_negotiation.cc:769] Sending connection context
0805 23:10:45.246145 (+ 83us) client_negotiation.cc:240] Negotiation successful
0805 23:10:45.246220 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"mutex_wait_us":23,"thread_start_us":131,"threads_started":1}
I20260805 23:10:45.246364 1083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:45.243015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.194:56903 (local address 127.25.92.193:44453)
0805 23:10:45.243136 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:45.243139 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:45.243158 (+ 19us) server_negotiation.cc:407] Connection header received
0805 23:10:45.243209 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:45.243211 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:45.243246 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:45.243338 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:10:45.243601 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:45.244091 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:45.245217 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:45.245821 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:45.245952 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:45.245999 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:45.246184 (+ 185us) server_negotiation.cc:299] Negotiation successful
0805 23:10:45.246261 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260805 23:10:45.246948 647 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f127de797db42249900d2dbb9e072bf" candidate_uuid: "fc9234f3fa864a598207c335d409c16c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64c076108dc34bc68cacff2bd1d9282f" is_pre_election: true
I20260805 23:10:45.247169 647 raft_consensus.cc:2454] T 4f127de797db42249900d2dbb9e072bf P 64c076108dc34bc68cacff2bd1d9282f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc9234f3fa864a598207c335d409c16c in term 0.
I20260805 23:10:45.247123 929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f127de797db42249900d2dbb9e072bf" candidate_uuid: "fc9234f3fa864a598207c335d409c16c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6569ceea53644f6b26a1e4c960fe340" is_pre_election: true
I20260805 23:10:45.247383 929 raft_consensus.cc:2454] T 4f127de797db42249900d2dbb9e072bf P e6569ceea53644f6b26a1e4c960fe340 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc9234f3fa864a598207c335d409c16c in term 0.
I20260805 23:10:45.247628 722 leader_election.cc:302] T 4f127de797db42249900d2dbb9e072bf P fc9234f3fa864a598207c335d409c16c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64c076108dc34bc68cacff2bd1d9282f, fc9234f3fa864a598207c335d409c16c; no voters:
I20260805 23:10:45.247946 1093 raft_consensus.cc:2781] T 4f127de797db42249900d2dbb9e072bf P fc9234f3fa864a598207c335d409c16c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260805 23:10:45.248000 1093 raft_consensus.cc:482] T 4f127de797db42249900d2dbb9e072bf P fc9234f3fa864a598207c335d409c16c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260805 23:10:45.248027 1093 raft_consensus.cc:3037] T 4f127de797db42249900d2dbb9e072bf P fc9234f3fa864a598207c335d409c16c [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:10:45.249182 1093 raft_consensus.cc:504] T 4f127de797db42249900d2dbb9e072bf P fc9234f3fa864a598207c335d409c16c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6569ceea53644f6b26a1e4c960fe340" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 41337 } } peers { permanent_uuid: "64c076108dc34bc68cacff2bd1d9282f" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 44453 } } peers { permanent_uuid: "fc9234f3fa864a598207c335d409c16c" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 36939 } }
I20260805 23:10:45.249552 1093 leader_election.cc:288] T 4f127de797db42249900d2dbb9e072bf P fc9234f3fa864a598207c335d409c16c [CANDIDATE]: Term 1 election: Requested vote from peers e6569ceea53644f6b26a1e4c960fe340 (127.25.92.195:41337), 64c076108dc34bc68cacff2bd1d9282f (127.25.92.193:44453)
I20260805 23:10:45.249995 929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f127de797db42249900d2dbb9e072bf" candidate_uuid: "fc9234f3fa864a598207c335d409c16c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6569ceea53644f6b26a1e4c960fe340"
I20260805 23:10:45.250123 929 raft_consensus.cc:3037] T 4f127de797db42249900d2dbb9e072bf P e6569ceea53644f6b26a1e4c960fe340 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:10:45.250154 647 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f127de797db42249900d2dbb9e072bf" candidate_uuid: "fc9234f3fa864a598207c335d409c16c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64c076108dc34bc68cacff2bd1d9282f"
I20260805 23:10:45.250324 647 raft_consensus.cc:3037] T 4f127de797db42249900d2dbb9e072bf P 64c076108dc34bc68cacff2bd1d9282f [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:10:45.251250 929 raft_consensus.cc:2454] T 4f127de797db42249900d2dbb9e072bf P e6569ceea53644f6b26a1e4c960fe340 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc9234f3fa864a598207c335d409c16c in term 1.
I20260805 23:10:45.251574 722 leader_election.cc:302] T 4f127de797db42249900d2dbb9e072bf P fc9234f3fa864a598207c335d409c16c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e6569ceea53644f6b26a1e4c960fe340, fc9234f3fa864a598207c335d409c16c; no voters:
I20260805 23:10:45.251603 647 raft_consensus.cc:2454] T 4f127de797db42249900d2dbb9e072bf P 64c076108dc34bc68cacff2bd1d9282f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc9234f3fa864a598207c335d409c16c in term 1.
I20260805 23:10:45.251772 1093 raft_consensus.cc:2781] T 4f127de797db42249900d2dbb9e072bf P fc9234f3fa864a598207c335d409c16c [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:10:45.252041 1093 raft_consensus.cc:686] T 4f127de797db42249900d2dbb9e072bf P fc9234f3fa864a598207c335d409c16c [term 1 LEADER]: Becoming Leader. State: Replica: fc9234f3fa864a598207c335d409c16c, State: Running, Role: LEADER
I20260805 23:10:45.252355 1093 consensus_queue.cc:237] T 4f127de797db42249900d2dbb9e072bf P fc9234f3fa864a598207c335d409c16c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6569ceea53644f6b26a1e4c960fe340" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 41337 } } peers { permanent_uuid: "64c076108dc34bc68cacff2bd1d9282f" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 44453 } } peers { permanent_uuid: "fc9234f3fa864a598207c335d409c16c" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 36939 } }
I20260805 23:10:45.253926 1094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:45.253679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:39541 (local address 127.25.92.194:35611)
0805 23:10:45.253805 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:45.253864 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:39541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260805 23:10:45.255496 322 catalog_manager.cc:5495] T 4f127de797db42249900d2dbb9e072bf P fc9234f3fa864a598207c335d409c16c reported cstate change: term changed from 0 to 1, leader changed from <none> to fc9234f3fa864a598207c335d409c16c (127.25.92.194). New cstate: current_term: 1 leader_uuid: "fc9234f3fa864a598207c335d409c16c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6569ceea53644f6b26a1e4c960fe340" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 41337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64c076108dc34bc68cacff2bd1d9282f" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 44453 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc9234f3fa864a598207c335d409c16c" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 36939 } health_report { overall_health: HEALTHY } } }
I20260805 23:10:45.273896 1084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:45.270246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60736 (local address 127.25.92.194:36939)
0805 23:10:45.270385 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:45.270390 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:45.270406 (+ 16us) server_negotiation.cc:407] Connection header received
0805 23:10:45.270506 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:45.270508 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:45.270564 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:45.270713 (+ 149us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 23:10:45.270962 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:45.271480 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:45.272328 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:45.272554 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:45.272674 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:45.272936 (+ 262us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 23:10:45.273400 (+ 464us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 23:10:45.273547 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:45.273654 (+ 107us) server_negotiation.cc:299] Negotiation successful
0805 23:10:45.273710 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260805 23:10:45.276928 647 raft_consensus.cc:1261] T 4f127de797db42249900d2dbb9e072bf P 64c076108dc34bc68cacff2bd1d9282f [term 1 FOLLOWER]: Refusing update from remote peer fc9234f3fa864a598207c335d409c16c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:10:45.277279 929 raft_consensus.cc:1261] T 4f127de797db42249900d2dbb9e072bf P e6569ceea53644f6b26a1e4c960fe340 [term 1 FOLLOWER]: Refusing update from remote peer fc9234f3fa864a598207c335d409c16c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:10:45.277544 1093 consensus_queue.cc:1035] T 4f127de797db42249900d2dbb9e072bf P fc9234f3fa864a598207c335d409c16c [LEADER]: Connected to new peer: Peer: permanent_uuid: "64c076108dc34bc68cacff2bd1d9282f" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 44453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:10:45.277884 1095 consensus_queue.cc:1035] T 4f127de797db42249900d2dbb9e072bf P fc9234f3fa864a598207c335d409c16c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6569ceea53644f6b26a1e4c960fe340" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 41337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:10:45.279743 1062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:45.279337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:39541 (local address 127.25.92.193:51477)
0805 23:10:45.279559 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:45.279644 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:39541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260805 23:10:45.279785 1065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:45.279501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:39541 (local address 127.25.92.195:51783)
0805 23:10:45.279657 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:45.279713 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:39541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260805 23:10:45.285327 1065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:45.284902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:39541 (local address 127.25.92.195:48675)
0805 23:10:45.285160 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:45.285224 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:39541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183}
I20260805 23:10:45.285876 1094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:45.284538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:39541 (local address 127.25.92.194:55839)
0805 23:10:45.285746 (+ 1208us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:45.285804 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:39541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1139}
I20260805 23:10:45.287606 1062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:45.287280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:39541 (local address 127.25.92.193:37179)
0805 23:10:45.287481 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:45.287534 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:39541: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125}
I20260805 23:10:45.293185 323 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57260:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:10:45.293370 323 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57260:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:10:45.297597 323 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d72d1f2dd8074bbbaf2fc4761feb7080: Sending DeleteTablet for 3 replicas of tablet 4f127de797db42249900d2dbb9e072bf
I20260805 23:10:45.298424 627 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f127de797db42249900d2dbb9e072bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:10:45 UTC) from {username='oryx', principal='oryx/127.25.92.253@KRBTEST.COM'} at 127.0.0.1:35156
I20260805 23:10:45.298766 909 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f127de797db42249900d2dbb9e072bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:10:45 UTC) from {username='oryx', principal='oryx/127.25.92.253@KRBTEST.COM'} at 127.0.0.1:35854
I20260805 23:10:45.299068 766 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f127de797db42249900d2dbb9e072bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:10:45 UTC) from {username='oryx', principal='oryx/127.25.92.253@KRBTEST.COM'} at 127.0.0.1:60728
I20260805 23:10:45.299590 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 471
I20260805 23:10:45.307117 1110 tablet_replica.cc:330] T 4f127de797db42249900d2dbb9e072bf P fc9234f3fa864a598207c335d409c16c: stopping tablet replica
I20260805 23:10:45.307134 1109 tablet_replica.cc:330] T 4f127de797db42249900d2dbb9e072bf P e6569ceea53644f6b26a1e4c960fe340: stopping tablet replica
I20260805 23:10:45.307297 1110 raft_consensus.cc:2229] T 4f127de797db42249900d2dbb9e072bf P fc9234f3fa864a598207c335d409c16c [term 1 LEADER]: Raft consensus shutting down.
I20260805 23:10:45.307484 1109 raft_consensus.cc:2229] T 4f127de797db42249900d2dbb9e072bf P e6569ceea53644f6b26a1e4c960fe340 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 23:10:45.307667 1110 raft_consensus.cc:2258] T 4f127de797db42249900d2dbb9e072bf P fc9234f3fa864a598207c335d409c16c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:10:45.307698 1109 raft_consensus.cc:2258] T 4f127de797db42249900d2dbb9e072bf P e6569ceea53644f6b26a1e4c960fe340 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:10:45.308686 1109 ts_tablet_manager.cc:1905] T 4f127de797db42249900d2dbb9e072bf P e6569ceea53644f6b26a1e4c960fe340: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:10:45.308710 1110 ts_tablet_manager.cc:1905] T 4f127de797db42249900d2dbb9e072bf P fc9234f3fa864a598207c335d409c16c: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260805 23:10:45.311316 1082 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:10:45.310503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.193:44453 (local address 127.0.0.1:35172)
0805 23:10:45.310635 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:45.310656 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:45.310762 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:45.311199 (+ 437us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.193:44453: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":35}
W20260805 23:10:45.311542 310 catalog_manager.cc:4584] TS 64c076108dc34bc68cacff2bd1d9282f (127.25.92.193:44453): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4f127de797db42249900d2dbb9e072bf: Network error: Client connection negotiation failed: client connection to 127.25.92.193:44453: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260805 23:10:45.312016 1109 ts_tablet_manager.cc:1918] T 4f127de797db42249900d2dbb9e072bf P e6569ceea53644f6b26a1e4c960fe340: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 23:10:45.312112 1109 log.cc:1196] T 4f127de797db42249900d2dbb9e072bf P e6569ceea53644f6b26a1e4c960fe340: Deleting WAL directory at /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971406251336-25971-0/minicluster-data/ts-2/wal/wals/4f127de797db42249900d2dbb9e072bf
I20260805 23:10:45.312119 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 702
I20260805 23:10:45.314400 1109 ts_tablet_manager.cc:1939] T 4f127de797db42249900d2dbb9e072bf P e6569ceea53644f6b26a1e4c960fe340: Deleting consensus metadata
I20260805 23:10:45.315228 310 catalog_manager.cc:4841] TS e6569ceea53644f6b26a1e4c960fe340 (127.25.92.195:41337): tablet 4f127de797db42249900d2dbb9e072bf (table test-table [id=a9bd7ac2837a4d48a8ba8ea6c0ab4060]) successfully deleted
I20260805 23:10:45.318944 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 841
I20260805 23:10:45.319199 1082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:45.318885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.194:36939 (local address 127.0.0.1:60748)
0805 23:10:45.319023 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:45.319096 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.194:36939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260805 23:10:45.326376 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 32648
I20260805 23:10:45.333139 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 32754
2026-08-05T23:10:45Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (7056 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 23:10:45 dist-test-slave-ss4n krb5kdc[1113](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 23:10:45 dist-test-slave-ss4n krb5kdc[1113](info): set up 2 sockets
Aug 05 23:10:45 dist-test-slave-ss4n krb5kdc[1113](info): commencing operation
krb5kdc: starting...
W20260805 23:10:46.488680 25971 mini_kdc.cc:121] Time spent starting KDC: real 1.129s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/krb5kdc/test-user.keytab.
Aug 05 23:10:46 dist-test-slave-ss4n krb5kdc[1113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971446, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T23:10:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:10:46Z Disabled control of system clock
WARNING: no policy specified for oryx/127.25.92.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.25.92.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.25.92.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/krb5kdc/oryx_127.25.92.254.keytab.
Entry for principal oryx/127.25.92.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/krb5kdc/oryx_127.25.92.254.keytab.
I20260805 23:10:46.618639 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.254:38875
--webserver_interface=127.25.92.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/krb5kdc/oryx_127.25.92.254.keytab
--principal=oryx/127.25.92.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:35893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.92.254:38875
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:10:46.736233 1126 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:10:46.736604 1126 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:10:46.736690 1126 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:10:46.740862 1126 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:10:46.740978 1126 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260805 23:10:46.741014 1126 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:10:46.741048 1126 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:10:46.741087 1126 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:10:46.741122 1126 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:10:46.746722 1126 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:35893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.92.254:38875
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/krb5kdc/oryx_127.25.92.254.keytab
--principal=oryx/127.25.92.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.254:38875
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.92.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1126
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:10:46.747902 1126 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:10:46.748965 1126 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:10:46.757572 1131 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:46.757571 1132 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:10:46.757620 1126 server_base.cc:1034] running on GCE node
W20260805 23:10:46.757576 1134 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:10:46.758477 1126 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:10:46.759624 1126 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:10:46.760867 1126 hybrid_clock.cc:648] HybridClock initialized: now 1785971446760855 us; error 34 us; skew 500 ppm
Aug 05 23:10:46 dist-test-slave-ss4n krb5kdc[1113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971446, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.92.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:10:46.763793 1126 init.cc:377] Logged in from keytab as oryx/127.25.92.254@KRBTEST.COM (short username oryx)
I20260805 23:10:46.764798 1126 webserver.cc:460] Webserver started at http://127.25.92.254:46623/ using document root <none> and password file <none>
I20260805 23:10:46.765621 1126 fs_manager.cc:359] Metadata directory not provided
I20260805 23:10:46.765715 1126 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:10:46.766002 1126 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:10:46.768157 1126 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/minicluster-data/master-0/data/instance:
uuid: "3d3f991025e246978b016ce9b762cfa0"
format_stamp: "Formatted at 2026-08-05 23:10:46 on dist-test-slave-ss4n"
server_key: "3ddd1cdf1ba6462e7930dbff728bb26f"
server_key_iv: "f7abda4c77bc86ee68e22a07ac7d7f8e"
server_key_version: "encryptionkey@0"
I20260805 23:10:46.768781 1126 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/minicluster-data/master-0/wal/instance:
uuid: "3d3f991025e246978b016ce9b762cfa0"
format_stamp: "Formatted at 2026-08-05 23:10:46 on dist-test-slave-ss4n"
server_key: "3ddd1cdf1ba6462e7930dbff728bb26f"
server_key_iv: "f7abda4c77bc86ee68e22a07ac7d7f8e"
server_key_version: "encryptionkey@0"
I20260805 23:10:46.772719 1126 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260805 23:10:46.775398 1143 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:46.776800 1126 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260805 23:10:46.776954 1126 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/minicluster-data/master-0/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/minicluster-data/master-0/wal
uuid: "3d3f991025e246978b016ce9b762cfa0"
format_stamp: "Formatted at 2026-08-05 23:10:46 on dist-test-slave-ss4n"
server_key: "3ddd1cdf1ba6462e7930dbff728bb26f"
server_key_iv: "f7abda4c77bc86ee68e22a07ac7d7f8e"
server_key_version: "encryptionkey@0"
I20260805 23:10:46.777081 1126 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:10:46.795490 1126 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:10:46.799134 1126 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:10:46.799358 1126 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:10:46.807965 1126 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.254:38875
I20260805 23:10:46.807981 1207 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.254:38875 every 8 connection(s)
I20260805 23:10:46.808949 1126 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
I20260805 23:10:46.812063 1208 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:10:46.815639 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 1126
I20260805 23:10:46.815797 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/minicluster-data/master-0/wal/instance
I20260805 23:10:46.816083 25971 external_mini_cluster.cc:1428] Setting key 17f736f5318c6c04531af1d558a19845
I20260805 23:10:46.819015 1208 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d3f991025e246978b016ce9b762cfa0: Bootstrap starting.
I20260805 23:10:46.822151 1208 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3d3f991025e246978b016ce9b762cfa0: Neither blocks nor log segments found. Creating new log.
I20260805 23:10:46.823104 1208 log.cc:824] T 00000000000000000000000000000000 P 3d3f991025e246978b016ce9b762cfa0: Log is configured to *not* fsync() on all Append() calls
I20260805 23:10:46.825706 1208 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d3f991025e246978b016ce9b762cfa0: No bootstrap required, opened a new log
Aug 05 23:10:46 dist-test-slave-ss4n krb5kdc[1113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971446, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM
I20260805 23:10:46.830813 1208 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3d3f991025e246978b016ce9b762cfa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d3f991025e246978b016ce9b762cfa0" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 38875 } }
I20260805 23:10:46.831096 1208 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3d3f991025e246978b016ce9b762cfa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:10:46.831161 1208 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3d3f991025e246978b016ce9b762cfa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d3f991025e246978b016ce9b762cfa0, State: Initialized, Role: FOLLOWER
I20260805 23:10:46.831736 1208 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3d3f991025e246978b016ce9b762cfa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d3f991025e246978b016ce9b762cfa0" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 38875 } }
I20260805 23:10:46.831945 1208 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3d3f991025e246978b016ce9b762cfa0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:10:46.832083 1208 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3d3f991025e246978b016ce9b762cfa0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:10:46.832221 1208 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3d3f991025e246978b016ce9b762cfa0 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:10:46.832988 1211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:46.817649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44162 (local address 127.25.92.254:38875)
0805 23:10:46.818118 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:46.818139 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:46.818185 (+ 46us) server_negotiation.cc:407] Connection header received
0805 23:10:46.819096 (+ 911us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:46.819122 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:46.819571 (+ 449us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:46.819960 (+ 389us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:46.820667 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:46.822431 (+ 1764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:46.823429 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:46.824090 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:46.824274 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:46.827454 (+ 3180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:46.827460 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:46.827475 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:46.827505 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:46.829618 (+ 2113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:46.830228 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:46.830238 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:46.830244 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:46.830371 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:46.830704 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:46.830706 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:46.830707 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:46.831245 (+ 538us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:46.831443 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:46.831767 (+ 324us) server_negotiation.cc:299] Negotiation successful
0805 23:10:46.832100 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":300,"thread_start_us":115,"threads_started":1}
I20260805 23:10:46.833437 1208 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3d3f991025e246978b016ce9b762cfa0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d3f991025e246978b016ce9b762cfa0" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 38875 } }
I20260805 23:10:46.833873 1208 leader_election.cc:302] T 00000000000000000000000000000000 P 3d3f991025e246978b016ce9b762cfa0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3d3f991025e246978b016ce9b762cfa0; no voters:
I20260805 23:10:46.834285 1208 leader_election.cc:288] T 00000000000000000000000000000000 P 3d3f991025e246978b016ce9b762cfa0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 23:10:46.834417 1213 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3d3f991025e246978b016ce9b762cfa0 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:10:46.834700 1213 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3d3f991025e246978b016ce9b762cfa0 [term 1 LEADER]: Becoming Leader. State: Replica: 3d3f991025e246978b016ce9b762cfa0, State: Running, Role: LEADER
I20260805 23:10:46.835067 1213 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3d3f991025e246978b016ce9b762cfa0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d3f991025e246978b016ce9b762cfa0" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 38875 } }
I20260805 23:10:46.835719 1208 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3d3f991025e246978b016ce9b762cfa0 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:10:46.837025 1214 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d3f991025e246978b016ce9b762cfa0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3d3f991025e246978b016ce9b762cfa0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d3f991025e246978b016ce9b762cfa0" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 38875 } } }
I20260805 23:10:46.837189 1214 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d3f991025e246978b016ce9b762cfa0 [sys.catalog]: This master's current role is: LEADER
I20260805 23:10:46.837515 1215 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d3f991025e246978b016ce9b762cfa0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3d3f991025e246978b016ce9b762cfa0. Latest consensus state: current_term: 1 leader_uuid: "3d3f991025e246978b016ce9b762cfa0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d3f991025e246978b016ce9b762cfa0" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 38875 } } }
I20260805 23:10:46.837630 1215 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d3f991025e246978b016ce9b762cfa0 [sys.catalog]: This master's current role is: LEADER
I20260805 23:10:46.837639 1223 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:10:46.841089 1223 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:10:46.847754 1223 catalog_manager.cc:1347] Generated new cluster ID: c2d444a9c4a74da1b6f328097feb2fed
I20260805 23:10:46.847853 1223 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:10:46.878927 1223 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:10:46.880302 1223 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:10:46.885627 1223 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3d3f991025e246978b016ce9b762cfa0: Generated new TSK 0
I20260805 23:10:46.886407 1223 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.25.92.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.25.92.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.25.92.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/krb5kdc/oryx_127.25.92.193.keytab.
Entry for principal oryx/127.25.92.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/krb5kdc/oryx_127.25.92.193.keytab.
I20260805 23:10:46.921404 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.92.193:0
--local_ip_for_outbound_sockets=127.25.92.193
--webserver_interface=127.25.92.193
--webserver_port=0
--tserver_master_addrs=127.25.92.254:38875
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/krb5kdc/oryx_127.25.92.193.keytab
--principal=oryx/127.25.92.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:35893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:10:47.058104 1235 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:10:47.058457 1235 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:10:47.058511 1235 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:10:47.062573 1235 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:10:47.062637 1235 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:10:47.062685 1235 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260805 23:10:47.062718 1235 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.92.193
I20260805 23:10:47.068053 1235 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:35893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/krb5kdc/oryx_127.25.92.193.keytab
--principal=oryx/127.25.92.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.92.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.92.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.25.92.254:38875
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1235
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.92.193
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:10:47.069243 1235 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:10:47.070276 1235 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:10:47.080330 1240 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:47.080317 1241 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:10:47.080456 1235 server_base.cc:1034] running on GCE node
W20260805 23:10:47.080282 1243 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:10:47.081270 1235 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:10:47.082086 1235 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:10:47.083299 1235 hybrid_clock.cc:648] HybridClock initialized: now 1785971447083261 us; error 52 us; skew 500 ppm
Aug 05 23:10:47 dist-test-slave-ss4n krb5kdc[1113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971447, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.92.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:10:47.086303 1235 init.cc:377] Logged in from keytab as oryx/127.25.92.193@KRBTEST.COM (short username oryx)
I20260805 23:10:47.087182 1235 webserver.cc:460] Webserver started at http://127.25.92.193:34987/ using document root <none> and password file <none>
I20260805 23:10:47.087761 1235 fs_manager.cc:359] Metadata directory not provided
I20260805 23:10:47.087823 1235 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:10:47.087994 1235 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:10:47.090103 1235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/minicluster-data/ts-0/data/instance:
uuid: "2e01c23563984d0682838b27189624fa"
format_stamp: "Formatted at 2026-08-05 23:10:47 on dist-test-slave-ss4n"
server_key: "a2620b79f4d9189ea4108e96535df42d"
server_key_iv: "fa2858811f336b0cce8a8e84069f02c4"
server_key_version: "encryptionkey@0"
I20260805 23:10:47.090637 1235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/minicluster-data/ts-0/wal/instance:
uuid: "2e01c23563984d0682838b27189624fa"
format_stamp: "Formatted at 2026-08-05 23:10:47 on dist-test-slave-ss4n"
server_key: "a2620b79f4d9189ea4108e96535df42d"
server_key_iv: "fa2858811f336b0cce8a8e84069f02c4"
server_key_version: "encryptionkey@0"
I20260805 23:10:47.094498 1235 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260805 23:10:47.096946 1250 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:47.098232 1235 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260805 23:10:47.098359 1235 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/minicluster-data/ts-0/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/minicluster-data/ts-0/wal
uuid: "2e01c23563984d0682838b27189624fa"
format_stamp: "Formatted at 2026-08-05 23:10:47 on dist-test-slave-ss4n"
server_key: "a2620b79f4d9189ea4108e96535df42d"
server_key_iv: "fa2858811f336b0cce8a8e84069f02c4"
server_key_version: "encryptionkey@0"
I20260805 23:10:47.098472 1235 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:10:47.115914 1235 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:10:47.119472 1235 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:10:47.119681 1235 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:10:47.121176 1235 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:10:47.121254 1235 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:47.121325 1235 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:10:47.121384 1235 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 05 23:10:47 dist-test-slave-ss4n krb5kdc[1113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971447, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.92.193@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM
I20260805 23:10:47.132617 1235 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.193:42565
I20260805 23:10:47.132680 1369 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.193:42565 every 8 connection(s)
I20260805 23:10:47.133630 1235 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/minicluster-data/ts-0/data/info.pb
I20260805 23:10:47.136315 1211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:47.122415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.193:42283 (local address 127.25.92.254:38875)
0805 23:10:47.122590 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:47.122596 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:47.123549 (+ 953us) server_negotiation.cc:407] Connection header received
0805 23:10:47.124967 (+ 1418us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:47.124971 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:47.125029 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:47.125136 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:47.126611 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:47.127214 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:47.128457 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:47.128726 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:47.128851 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:47.132039 (+ 3188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:47.132043 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:47.132046 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:47.132084 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:47.134177 (+ 2093us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:47.134815 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:47.134818 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:47.134820 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:47.134907 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:47.135265 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:47.135270 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:47.135273 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:47.135504 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:47.135585 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:47.136062 (+ 477us) server_negotiation.cc:299] Negotiation successful
0805 23:10:47.136191 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260805 23:10:47.137238 1259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:47.122863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:38875 (local address 127.25.92.193:42283)
0805 23:10:47.123355 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:47.123410 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:47.124694 (+ 1284us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:47.125343 (+ 649us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:47.125350 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:47.125935 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:47.126435 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:47.126450 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:47.127358 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:47.127362 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:47.128315 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:47.128325 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:47.128915 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:47.128920 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:47.129112 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:47.129973 (+ 861us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:10:47.129990 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:47.131838 (+ 1848us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:10:47.134324 (+ 2486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:47.134330 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:47.134344 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:47.134705 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:47.135002 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:47.135005 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:47.135006 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:47.135135 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:47.135591 (+ 456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:10:47.135596 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:10:47.135832 (+ 236us) client_negotiation.cc:769] Sending connection context
0805 23:10:47.136003 (+ 171us) client_negotiation.cc:240] Negotiation successful
0805 23:10:47.136271 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":152,"threads_started":1}
I20260805 23:10:47.138599 1370 heartbeater.cc:347] Connected to a master server at 127.25.92.254:38875
I20260805 23:10:47.138744 1370 heartbeater.cc:464] Registering TS with master...
I20260805 23:10:47.139356 1370 heartbeater.cc:511] Master 127.25.92.254:38875 requested a full tablet report, sending...
I20260805 23:10:47.139772 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 1235
I20260805 23:10:47.139925 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/minicluster-data/ts-0/wal/instance
I20260805 23:10:47.140282 25971 external_mini_cluster.cc:1428] Setting key 88482153def332b48e3aa4bc7977de07
I20260805 23:10:47.141049 1162 ts_manager.cc:194] Registered new tserver with Master: 2e01c23563984d0682838b27189624fa (127.25.92.193:42565)
I20260805 23:10:47.142490 1162 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.25.92.193@KRBTEST.COM'} at 127.25.92.193:42283
I20260805 23:10:47.150491 1211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:47.141025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.193:36219 (local address 127.25.92.254:38875)
0805 23:10:47.141174 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:47.141180 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:47.142399 (+ 1219us) server_negotiation.cc:407] Connection header received
0805 23:10:47.142519 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:47.142522 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:47.142563 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:47.142652 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:47.143215 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:47.143945 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:47.145321 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:47.145618 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:47.145703 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:47.147096 (+ 1393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:47.147099 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:47.147101 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:47.147125 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:47.148749 (+ 1624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:47.149270 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:47.149273 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:47.149274 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:47.149349 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:47.149681 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:47.149683 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:47.149684 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:47.149857 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:47.149937 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:47.150266 (+ 329us) server_negotiation.cc:299] Negotiation successful
0805 23:10:47.150368 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260805 23:10:47.150626 1376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:47.140963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:38875 (local address 127.25.92.193:36219)
0805 23:10:47.142308 (+ 1345us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:47.142330 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:47.142441 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:47.142766 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:47.142768 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:47.142941 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:47.143069 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:47.143077 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:47.144104 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:47.144111 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:47.145189 (+ 1078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:47.145202 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:47.145740 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:47.145747 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:47.145880 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:47.146422 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:10:47.146442 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:47.146952 (+ 510us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:10:47.148871 (+ 1919us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:47.148878 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:47.148882 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:47.149171 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:47.149440 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:47.149445 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:47.149450 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:47.149580 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:47.149963 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:10:47.149967 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:10:47.150080 (+ 113us) client_negotiation.cc:769] Sending connection context
0805 23:10:47.150207 (+ 127us) client_negotiation.cc:240] Negotiation successful
0805 23:10:47.150372 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1251,"mutex_wait_us":1,"thread_start_us":106,"threads_started":1}
WARNING: no policy specified for oryx/127.25.92.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.25.92.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.25.92.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/krb5kdc/oryx_127.25.92.194.keytab.
Entry for principal oryx/127.25.92.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/krb5kdc/oryx_127.25.92.194.keytab.
I20260805 23:10:47.173328 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.92.194:0
--local_ip_for_outbound_sockets=127.25.92.194
--webserver_interface=127.25.92.194
--webserver_port=0
--tserver_master_addrs=127.25.92.254:38875
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/krb5kdc/oryx_127.25.92.194.keytab
--principal=oryx/127.25.92.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:35893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:10:47.291290 1379 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:10:47.291563 1379 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:10:47.291616 1379 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:10:47.295696 1379 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:10:47.295773 1379 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:10:47.295814 1379 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260805 23:10:47.295847 1379 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.92.194
I20260805 23:10:47.301208 1379 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:35893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/krb5kdc/oryx_127.25.92.194.keytab
--principal=oryx/127.25.92.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.92.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.92.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.25.92.254:38875
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1379
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.92.194
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:10:47.302513 1379 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:10:47.303506 1379 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260805 23:10:47.313274 1379 server_base.cc:1034] running on GCE node
W20260805 23:10:47.313288 1385 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:47.313320 1384 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:47.313572 1387 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:10:47.313992 1379 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:10:47.314604 1379 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:10:47.315815 1379 hybrid_clock.cc:648] HybridClock initialized: now 1785971447315800 us; error 29 us; skew 500 ppm
Aug 05 23:10:47 dist-test-slave-ss4n krb5kdc[1113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971447, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.92.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:10:47.318758 1379 init.cc:377] Logged in from keytab as oryx/127.25.92.194@KRBTEST.COM (short username oryx)
I20260805 23:10:47.319711 1379 webserver.cc:460] Webserver started at http://127.25.92.194:42745/ using document root <none> and password file <none>
I20260805 23:10:47.320253 1379 fs_manager.cc:359] Metadata directory not provided
I20260805 23:10:47.320313 1379 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:10:47.320480 1379 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:10:47.322603 1379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/minicluster-data/ts-1/data/instance:
uuid: "599840b07db24ad088880d0fdc1c5cba"
format_stamp: "Formatted at 2026-08-05 23:10:47 on dist-test-slave-ss4n"
server_key: "2b8e4b3a0cf7940db228483f554cf7fb"
server_key_iv: "fa8a95631c1c9e636194ec4164fcf244"
server_key_version: "encryptionkey@0"
I20260805 23:10:47.323148 1379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/minicluster-data/ts-1/wal/instance:
uuid: "599840b07db24ad088880d0fdc1c5cba"
format_stamp: "Formatted at 2026-08-05 23:10:47 on dist-test-slave-ss4n"
server_key: "2b8e4b3a0cf7940db228483f554cf7fb"
server_key_iv: "fa8a95631c1c9e636194ec4164fcf244"
server_key_version: "encryptionkey@0"
I20260805 23:10:47.327076 1379 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260805 23:10:47.329710 1394 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:47.331189 1379 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:10:47.331300 1379 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/minicluster-data/ts-1/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/minicluster-data/ts-1/wal
uuid: "599840b07db24ad088880d0fdc1c5cba"
format_stamp: "Formatted at 2026-08-05 23:10:47 on dist-test-slave-ss4n"
server_key: "2b8e4b3a0cf7940db228483f554cf7fb"
server_key_iv: "fa8a95631c1c9e636194ec4164fcf244"
server_key_version: "encryptionkey@0"
I20260805 23:10:47.331401 1379 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:10:47.362104 1379 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:10:47.365617 1379 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:10:47.365789 1379 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:10:47.367281 1379 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:10:47.367336 1379 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:47.367376 1379 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:10:47.367393 1379 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 05 23:10:47 dist-test-slave-ss4n krb5kdc[1113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971447, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.92.194@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM
I20260805 23:10:47.378878 1379 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.194:44967
I20260805 23:10:47.378959 1513 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.194:44967 every 8 connection(s)
I20260805 23:10:47.379807 1379 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/minicluster-data/ts-1/data/info.pb
I20260805 23:10:47.380762 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 1379
I20260805 23:10:47.380888 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/minicluster-data/ts-1/wal/instance
I20260805 23:10:47.381224 25971 external_mini_cluster.cc:1428] Setting key 01a4611026ddbe27980262157f66ddd1
I20260805 23:10:47.382370 1211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:47.368211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.194:56149 (local address 127.25.92.254:38875)
0805 23:10:47.368335 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:47.368339 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:47.369393 (+ 1054us) server_negotiation.cc:407] Connection header received
0805 23:10:47.370533 (+ 1140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:47.370538 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:47.370653 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:47.370720 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:47.372108 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:47.373025 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:47.374486 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:47.374781 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:47.374904 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:47.378225 (+ 3321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:47.378229 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:47.378231 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:47.378259 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:47.379894 (+ 1635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:47.380570 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:47.380576 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:47.380580 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:47.380674 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:47.381087 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:47.381090 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:47.381091 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:47.381335 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:47.381458 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:47.382070 (+ 612us) server_negotiation.cc:299] Negotiation successful
0805 23:10:47.382223 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260805 23:10:47.383255 1404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:47.368764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:38875 (local address 127.25.92.194:56149)
0805 23:10:47.369247 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:47.369289 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:47.370280 (+ 991us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:47.371045 (+ 765us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:47.371050 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:47.371364 (+ 314us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:47.371939 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:47.371956 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:47.373197 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:47.373200 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:47.374367 (+ 1167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:47.374381 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:47.374954 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:47.374958 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:47.375137 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:47.375809 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:10:47.375836 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:47.378036 (+ 2200us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:10:47.380040 (+ 2004us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:47.380046 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:47.380059 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:47.380430 (+ 371us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:47.380796 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:47.380800 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:47.380802 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:47.380988 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:47.381456 (+ 468us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:10:47.381462 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:10:47.381790 (+ 328us) client_negotiation.cc:769] Sending connection context
0805 23:10:47.381981 (+ 191us) client_negotiation.cc:240] Negotiation successful
0805 23:10:47.382255 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":141,"threads_started":1}
I20260805 23:10:47.385913 1514 heartbeater.cc:347] Connected to a master server at 127.25.92.254:38875
I20260805 23:10:47.386072 1514 heartbeater.cc:464] Registering TS with master...
I20260805 23:10:47.386667 1514 heartbeater.cc:511] Master 127.25.92.254:38875 requested a full tablet report, sending...
I20260805 23:10:47.387781 1162 ts_manager.cc:194] Registered new tserver with Master: 599840b07db24ad088880d0fdc1c5cba (127.25.92.194:44967)
I20260805 23:10:47.388567 1162 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.25.92.194@KRBTEST.COM'} at 127.25.92.194:56149
WARNING: no policy specified for oryx/127.25.92.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.25.92.195@KRBTEST.COM" created.
I20260805 23:10:47.395460 1211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:47.388112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.194:60089 (local address 127.25.92.254:38875)
0805 23:10:47.388224 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:47.388228 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:47.388504 (+ 276us) server_negotiation.cc:407] Connection header received
0805 23:10:47.388615 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:47.388621 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:47.388693 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:47.388790 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:47.389315 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:47.389890 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:47.390744 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:47.390973 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:47.391087 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:47.392188 (+ 1101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:47.392191 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:47.392195 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:47.392219 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:47.393796 (+ 1577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:47.394211 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:47.394215 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:47.394217 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:47.394304 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:47.394587 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:47.394590 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:47.394592 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:47.394817 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:47.394927 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:47.395184 (+ 257us) server_negotiation.cc:299] Negotiation successful
0805 23:10:47.395297 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260805 23:10:47.395473 1521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:47.388142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:38875 (local address 127.25.92.194:60089)
0805 23:10:47.388405 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:47.388418 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:47.388514 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:47.388822 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:47.388828 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:47.389068 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:47.389193 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:47.389200 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:47.390013 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:47.390016 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:47.390617 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:47.390625 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:47.391084 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:47.391087 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:47.391176 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:47.391717 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:10:47.391735 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:47.392071 (+ 336us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:10:47.393908 (+ 1837us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:47.393913 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:47.393916 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:47.394114 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:47.394402 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:47.394405 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:47.394407 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:47.394492 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:47.394916 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:10:47.394919 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:10:47.395018 (+ 99us) client_negotiation.cc:769] Sending connection context
0805 23:10:47.395127 (+ 109us) client_negotiation.cc:240] Negotiation successful
0805 23:10:47.395285 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"mutex_wait_us":7,"thread_start_us":102,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.25.92.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/krb5kdc/oryx_127.25.92.195.keytab.
Entry for principal oryx/127.25.92.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/krb5kdc/oryx_127.25.92.195.keytab.
I20260805 23:10:47.413758 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.92.195:0
--local_ip_for_outbound_sockets=127.25.92.195
--webserver_interface=127.25.92.195
--webserver_port=0
--tserver_master_addrs=127.25.92.254:38875
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/krb5kdc/oryx_127.25.92.195.keytab
--principal=oryx/127.25.92.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:35893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:10:47.527055 1523 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:10:47.527366 1523 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:10:47.527442 1523 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:10:47.531770 1523 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:10:47.531862 1523 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:10:47.531937 1523 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260805 23:10:47.532030 1523 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.92.195
I20260805 23:10:47.538085 1523 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:35893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/krb5kdc/oryx_127.25.92.195.keytab
--principal=oryx/127.25.92.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.92.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.92.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.25.92.254:38875
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1523
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.92.195
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:10:47.539228 1523 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:10:47.540316 1523 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:10:47.551138 1528 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:47.551151 1529 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:47.551237 1531 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:10:47.551326 1523 server_base.cc:1034] running on GCE node
I20260805 23:10:47.551862 1523 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:10:47.552568 1523 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:10:47.553768 1523 hybrid_clock.cc:648] HybridClock initialized: now 1785971447553753 us; error 39 us; skew 500 ppm
Aug 05 23:10:47 dist-test-slave-ss4n krb5kdc[1113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971447, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.92.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:10:47.556612 1523 init.cc:377] Logged in from keytab as oryx/127.25.92.195@KRBTEST.COM (short username oryx)
I20260805 23:10:47.557633 1523 webserver.cc:460] Webserver started at http://127.25.92.195:42261/ using document root <none> and password file <none>
I20260805 23:10:47.558276 1523 fs_manager.cc:359] Metadata directory not provided
I20260805 23:10:47.558370 1523 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:10:47.558614 1523 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:10:47.560757 1523 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/minicluster-data/ts-2/data/instance:
uuid: "b2c00b8f050b47fabe1abc7847e2db48"
format_stamp: "Formatted at 2026-08-05 23:10:47 on dist-test-slave-ss4n"
server_key: "db50f109a75a1e3d8a3eafd07431f1a8"
server_key_iv: "57440d11864478c1af78e4e78019c336"
server_key_version: "encryptionkey@0"
I20260805 23:10:47.561350 1523 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/minicluster-data/ts-2/wal/instance:
uuid: "b2c00b8f050b47fabe1abc7847e2db48"
format_stamp: "Formatted at 2026-08-05 23:10:47 on dist-test-slave-ss4n"
server_key: "db50f109a75a1e3d8a3eafd07431f1a8"
server_key_iv: "57440d11864478c1af78e4e78019c336"
server_key_version: "encryptionkey@0"
I20260805 23:10:47.565117 1523 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.003s
I20260805 23:10:47.567780 1538 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:47.569051 1523 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260805 23:10:47.569218 1523 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/minicluster-data/ts-2/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/minicluster-data/ts-2/wal
uuid: "b2c00b8f050b47fabe1abc7847e2db48"
format_stamp: "Formatted at 2026-08-05 23:10:47 on dist-test-slave-ss4n"
server_key: "db50f109a75a1e3d8a3eafd07431f1a8"
server_key_iv: "57440d11864478c1af78e4e78019c336"
server_key_version: "encryptionkey@0"
I20260805 23:10:47.569337 1523 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:10:47.585371 1523 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:10:47.588490 1523 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:10:47.588696 1523 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:10:47.590366 1523 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:10:47.590440 1523 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:47.590507 1523 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:10:47.590591 1523 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
Aug 05 23:10:47 dist-test-slave-ss4n krb5kdc[1113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971447, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.92.195@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM
I20260805 23:10:47.601665 1523 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.195:40101
I20260805 23:10:47.601727 1657 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.195:40101 every 8 connection(s)
I20260805 23:10:47.602716 1523 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/minicluster-data/ts-2/data/info.pb
I20260805 23:10:47.604866 1211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:47.591077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.195:46331 (local address 127.25.92.254:38875)
0805 23:10:47.591245 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:47.591251 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:47.592099 (+ 848us) server_negotiation.cc:407] Connection header received
0805 23:10:47.593042 (+ 943us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:47.593045 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:47.593103 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:47.593211 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:47.594474 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:47.595219 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:47.596593 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:47.596826 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:47.596962 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:47.600832 (+ 3870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:47.600835 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:47.600838 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:47.600867 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:47.602609 (+ 1742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:47.603253 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:47.603256 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:47.603258 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:47.603352 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:47.603731 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:47.603735 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:47.603737 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:47.603940 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:47.604058 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:47.604555 (+ 497us) server_negotiation.cc:299] Negotiation successful
0805 23:10:47.604694 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260805 23:10:47.605670 1547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:47.591472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:38875 (local address 127.25.92.195:46331)
0805 23:10:47.591927 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:47.591976 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:47.592822 (+ 846us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:47.593381 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:47.593391 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:47.593879 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:47.594325 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:47.594338 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:47.595344 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:47.595347 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:47.596475 (+ 1128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:47.596488 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:47.596993 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:47.596997 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:47.597226 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:47.597987 (+ 761us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:10:47.598010 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:47.600656 (+ 2646us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:10:47.602726 (+ 2070us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:47.602732 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:47.602751 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:47.603120 (+ 369us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:47.603474 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:47.603478 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:47.603480 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:47.603624 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:47.604067 (+ 443us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:10:47.604075 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:10:47.604317 (+ 242us) client_negotiation.cc:769] Sending connection context
0805 23:10:47.604507 (+ 190us) client_negotiation.cc:240] Negotiation successful
0805 23:10:47.604818 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":128,"threads_started":1}
I20260805 23:10:47.606909 1658 heartbeater.cc:347] Connected to a master server at 127.25.92.254:38875
I20260805 23:10:47.607069 1658 heartbeater.cc:464] Registering TS with master...
I20260805 23:10:47.607615 1658 heartbeater.cc:511] Master 127.25.92.254:38875 requested a full tablet report, sending...
I20260805 23:10:47.608831 1158 ts_manager.cc:194] Registered new tserver with Master: b2c00b8f050b47fabe1abc7847e2db48 (127.25.92.195:40101)
I20260805 23:10:47.609840 1158 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.25.92.195@KRBTEST.COM'} at 127.25.92.195:46331
I20260805 23:10:47.611089 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 1523
I20260805 23:10:47.611222 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/minicluster-data/ts-2/wal/instance
I20260805 23:10:47.611552 25971 external_mini_cluster.cc:1428] Setting key f17adb238d703417a01485fa5e1bdb82
I20260805 23:10:47.613945 25971 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260805 23:10:47.618052 1211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:47.609551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.195:39143 (local address 127.25.92.254:38875)
0805 23:10:47.609692 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:47.609697 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:47.609715 (+ 18us) server_negotiation.cc:407] Connection header received
0805 23:10:47.609874 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:47.609878 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:47.609929 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:47.610027 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:47.610691 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:47.611551 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:47.612542 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:47.612881 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:47.612994 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:47.614437 (+ 1443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:47.614442 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:47.614445 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:47.614473 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:47.616309 (+ 1836us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:47.616757 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:47.616763 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:47.616768 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:47.616862 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:47.617158 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:47.617164 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:47.617168 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:47.617361 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:47.617472 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:47.617817 (+ 345us) server_negotiation.cc:299] Negotiation successful
0805 23:10:47.617922 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260805 23:10:47.618223 1664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:47.609250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:38875 (local address 127.25.92.195:39143)
0805 23:10:47.609634 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:47.609658 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:47.609760 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:47.610183 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:47.610188 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:47.610437 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:47.610564 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:47.610571 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:47.611711 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:47.611715 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:47.612436 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:47.612447 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:47.613025 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:47.613029 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:47.613128 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:47.613840 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:10:47.613859 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:47.614289 (+ 430us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:10:47.616441 (+ 2152us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:47.616445 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:47.616447 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:47.616653 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:47.616970 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:47.616974 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:47.616976 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:47.617065 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:47.617460 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:10:47.617463 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:10:47.617625 (+ 162us) client_negotiation.cc:769] Sending connection context
0805 23:10:47.617743 (+ 118us) client_negotiation.cc:240] Negotiation successful
0805 23:10:47.617922 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":118,"threads_started":1}
Aug 05 23:10:47 dist-test-slave-ss4n krb5kdc[1113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971447, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 05 23:10:47 dist-test-slave-ss4n krb5kdc[1113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:10:47 dist-test-slave-ss4n krb5kdc[1113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
W20260805 23:10:47.641371 1211 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:10:47.633688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44166 (local address 127.25.92.254:38875)
0805 23:10:47.633817 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:47.633821 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:47.633860 (+ 39us) server_negotiation.cc:407] Connection header received
0805 23:10:47.633907 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:47.633909 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:47.633956 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:47.634037 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:47.634850 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:47.635472 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:47.636542 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:47.636831 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:47.636975 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:47.641134 (+ 4159us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44166: BlockingRecv error: recv got EOF from 127.0.0.1:44166 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 05 23:10:47 dist-test-slave-ss4n krb5kdc[1113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971447, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM
I20260805 23:10:47.651302 1211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:47.642937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44170 (local address 127.25.92.254:38875)
0805 23:10:47.643070 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:47.643074 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:47.643177 (+ 103us) server_negotiation.cc:407] Connection header received
0805 23:10:47.643349 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:47.643354 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:47.643418 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:47.643530 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:47.644202 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:47.644725 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:47.645545 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:47.645774 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:47.645888 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:47.648215 (+ 2327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:47.648223 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:47.648225 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:47.648249 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:47.649656 (+ 1407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:47.650068 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:47.650073 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:47.650075 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:47.650165 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:47.650439 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:47.650443 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:47.650445 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:47.650637 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:47.650782 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:47.651004 (+ 222us) server_negotiation.cc:299] Negotiation successful
0805 23:10:47.651122 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260805 23:10:47.654400 1158 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44170:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260805 23:10:47.657318 1158 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260805 23:10:47.672147 1684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:47.666458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47526 (local address 127.25.92.193:42565)
0805 23:10:47.667131 (+ 673us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:47.667138 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:47.667158 (+ 20us) server_negotiation.cc:407] Connection header received
0805 23:10:47.667228 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:47.667232 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:47.667413 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:47.667949 (+ 536us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:10:47.668073 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:47.669102 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:47.670609 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:47.671414 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:47.671534 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:47.671685 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:47.671896 (+ 211us) server_negotiation.cc:299] Negotiation successful
0805 23:10:47.671997 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":567,"thread_start_us":89,"threads_started":1}
I20260805 23:10:47.672122 1680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:47.665788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.193:42565 (local address 127.0.0.1:47526)
0805 23:10:47.666618 (+ 830us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:47.666633 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:47.666724 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:47.667672 (+ 948us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:47.667676 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:47.667711 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:10:47.667804 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:47.667810 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:47.669208 (+ 1398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:47.669214 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:47.670452 (+ 1238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:47.670461 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:47.671599 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:47.671602 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:47.671701 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:47.671735 (+ 34us) client_negotiation.cc:769] Sending connection context
0805 23:10:47.671850 (+ 115us) client_negotiation.cc:240] Negotiation successful
0805 23:10:47.671969 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":708,"mutex_wait_us":169,"spinlock_wait_cycles":768,"thread_start_us":89,"threads_started":1}
I20260805 23:10:47.672771 1683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:47.666385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.194:44967 (local address 127.0.0.1:60674)
0805 23:10:47.666682 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:47.666694 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:47.666762 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:47.667654 (+ 892us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:47.667657 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:47.667674 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:10:47.667789 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:47.667798 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:47.668754 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:47.668760 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:47.670446 (+ 1686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:47.670452 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:47.672285 (+ 1833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:47.672289 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:47.672394 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:47.672427 (+ 33us) client_negotiation.cc:769] Sending connection context
0805 23:10:47.672507 (+ 80us) client_negotiation.cc:240] Negotiation successful
0805 23:10:47.672589 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"mutex_wait_us":125,"spinlock_wait_cycles":9472,"thread_start_us":93,"threads_started":1}
I20260805 23:10:47.673240 1685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:47.666803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60674 (local address 127.25.92.194:44967)
0805 23:10:47.667248 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:47.667259 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:47.667280 (+ 21us) server_negotiation.cc:407] Connection header received
0805 23:10:47.667346 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:47.667351 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:47.667537 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:47.667685 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:10:47.667911 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:47.668621 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:47.670573 (+ 1952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:47.671414 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:47.671539 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:47.671707 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:47.672982 (+ 1275us) server_negotiation.cc:299] Negotiation successful
0805 23:10:47.673097 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":129,"threads_started":1}
I20260805 23:10:47.674108 1681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:47.665691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.195:40101 (local address 127.0.0.1:34862)
0805 23:10:47.666172 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:47.666218 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:47.666480 (+ 262us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:47.667270 (+ 790us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:47.667275 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:47.667301 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:10:47.667439 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:47.667446 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:47.668944 (+ 1498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:47.668948 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:47.670435 (+ 1487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:47.670446 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:47.673653 (+ 3207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:47.673657 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:47.673765 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:47.673791 (+ 26us) client_negotiation.cc:769] Sending connection context
0805 23:10:47.673885 (+ 94us) client_negotiation.cc:240] Negotiation successful
0805 23:10:47.673959 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"mutex_wait_us":78,"thread_start_us":95,"threads_started":1}
I20260805 23:10:47.674480 1682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:47.665821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34862 (local address 127.25.92.195:40101)
0805 23:10:47.666365 (+ 544us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:47.666388 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:47.666416 (+ 28us) server_negotiation.cc:407] Connection header received
0805 23:10:47.666697 (+ 281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:47.666706 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:47.666931 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:47.667079 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:10:47.667597 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:47.668831 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:47.671769 (+ 2938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:47.672642 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:47.673651 (+ 1009us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:47.673820 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:47.674241 (+ 421us) server_negotiation.cc:299] Negotiation successful
0805 23:10:47.674366 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":100,"threads_started":1}
I20260805 23:10:47.675834 1446 tablet_service.cc:1467] Processing CreateTablet for tablet f66c54d560644233a330b92df87046c6 (DEFAULT_TABLE table=test-table [id=10ef581f78844329b91c97882ba14453]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:10:47.675832 1303 tablet_service.cc:1467] Processing CreateTablet for tablet f66c54d560644233a330b92df87046c6 (DEFAULT_TABLE table=test-table [id=10ef581f78844329b91c97882ba14453]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:10:47.677173 1303 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f66c54d560644233a330b92df87046c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:10:47.677173 1446 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f66c54d560644233a330b92df87046c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:10:47.677433 1590 tablet_service.cc:1467] Processing CreateTablet for tablet f66c54d560644233a330b92df87046c6 (DEFAULT_TABLE table=test-table [id=10ef581f78844329b91c97882ba14453]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:10:47.678648 1590 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f66c54d560644233a330b92df87046c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:10:47.685057 1686 tablet_bootstrap.cc:492] T f66c54d560644233a330b92df87046c6 P 2e01c23563984d0682838b27189624fa: Bootstrap starting.
I20260805 23:10:47.686642 1688 tablet_bootstrap.cc:492] T f66c54d560644233a330b92df87046c6 P b2c00b8f050b47fabe1abc7847e2db48: Bootstrap starting.
I20260805 23:10:47.687887 1686 tablet_bootstrap.cc:654] T f66c54d560644233a330b92df87046c6 P 2e01c23563984d0682838b27189624fa: Neither blocks nor log segments found. Creating new log.
I20260805 23:10:47.688515 1688 tablet_bootstrap.cc:654] T f66c54d560644233a330b92df87046c6 P b2c00b8f050b47fabe1abc7847e2db48: Neither blocks nor log segments found. Creating new log.
I20260805 23:10:47.688865 1686 log.cc:824] T f66c54d560644233a330b92df87046c6 P 2e01c23563984d0682838b27189624fa: Log is configured to *not* fsync() on all Append() calls
I20260805 23:10:47.689363 1688 log.cc:824] T f66c54d560644233a330b92df87046c6 P b2c00b8f050b47fabe1abc7847e2db48: Log is configured to *not* fsync() on all Append() calls
I20260805 23:10:47.690275 1687 tablet_bootstrap.cc:492] T f66c54d560644233a330b92df87046c6 P 599840b07db24ad088880d0fdc1c5cba: Bootstrap starting.
I20260805 23:10:47.691161 1686 tablet_bootstrap.cc:492] T f66c54d560644233a330b92df87046c6 P 2e01c23563984d0682838b27189624fa: No bootstrap required, opened a new log
I20260805 23:10:47.691418 1686 ts_tablet_manager.cc:1397] T f66c54d560644233a330b92df87046c6 P 2e01c23563984d0682838b27189624fa: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260805 23:10:47.692386 1687 tablet_bootstrap.cc:654] T f66c54d560644233a330b92df87046c6 P 599840b07db24ad088880d0fdc1c5cba: Neither blocks nor log segments found. Creating new log.
I20260805 23:10:47.693111 1687 log.cc:824] T f66c54d560644233a330b92df87046c6 P 599840b07db24ad088880d0fdc1c5cba: Log is configured to *not* fsync() on all Append() calls
I20260805 23:10:47.694918 1687 tablet_bootstrap.cc:492] T f66c54d560644233a330b92df87046c6 P 599840b07db24ad088880d0fdc1c5cba: No bootstrap required, opened a new log
I20260805 23:10:47.695142 1687 ts_tablet_manager.cc:1397] T f66c54d560644233a330b92df87046c6 P 599840b07db24ad088880d0fdc1c5cba: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.000s
I20260805 23:10:47.695194 1688 tablet_bootstrap.cc:492] T f66c54d560644233a330b92df87046c6 P b2c00b8f050b47fabe1abc7847e2db48: No bootstrap required, opened a new log
I20260805 23:10:47.695765 1688 ts_tablet_manager.cc:1397] T f66c54d560644233a330b92df87046c6 P b2c00b8f050b47fabe1abc7847e2db48: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.000s
I20260805 23:10:47.696436 1686 raft_consensus.cc:348] T f66c54d560644233a330b92df87046c6 P 2e01c23563984d0682838b27189624fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2c00b8f050b47fabe1abc7847e2db48" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 40101 } } peers { permanent_uuid: "2e01c23563984d0682838b27189624fa" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 42565 } } peers { permanent_uuid: "599840b07db24ad088880d0fdc1c5cba" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 44967 } }
I20260805 23:10:47.696795 1686 raft_consensus.cc:374] T f66c54d560644233a330b92df87046c6 P 2e01c23563984d0682838b27189624fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:10:47.696911 1686 raft_consensus.cc:729] T f66c54d560644233a330b92df87046c6 P 2e01c23563984d0682838b27189624fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e01c23563984d0682838b27189624fa, State: Initialized, Role: FOLLOWER
I20260805 23:10:47.697468 1686 consensus_queue.cc:260] T f66c54d560644233a330b92df87046c6 P 2e01c23563984d0682838b27189624fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2c00b8f050b47fabe1abc7847e2db48" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 40101 } } peers { permanent_uuid: "2e01c23563984d0682838b27189624fa" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 42565 } } peers { permanent_uuid: "599840b07db24ad088880d0fdc1c5cba" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 44967 } }
I20260805 23:10:47.698755 1686 ts_tablet_manager.cc:1428] T f66c54d560644233a330b92df87046c6 P 2e01c23563984d0682838b27189624fa: Time spent starting tablet: real 0.007s user 0.004s sys 0.004s
I20260805 23:10:47.699210 1370 heartbeater.cc:503] Master 127.25.92.254:38875 was elected leader, sending a full tablet report...
I20260805 23:10:47.699043 1687 raft_consensus.cc:348] T f66c54d560644233a330b92df87046c6 P 599840b07db24ad088880d0fdc1c5cba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2c00b8f050b47fabe1abc7847e2db48" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 40101 } } peers { permanent_uuid: "2e01c23563984d0682838b27189624fa" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 42565 } } peers { permanent_uuid: "599840b07db24ad088880d0fdc1c5cba" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 44967 } }
I20260805 23:10:47.699322 1687 raft_consensus.cc:374] T f66c54d560644233a330b92df87046c6 P 599840b07db24ad088880d0fdc1c5cba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:10:47.699370 1687 raft_consensus.cc:729] T f66c54d560644233a330b92df87046c6 P 599840b07db24ad088880d0fdc1c5cba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 599840b07db24ad088880d0fdc1c5cba, State: Initialized, Role: FOLLOWER
I20260805 23:10:47.699810 1687 consensus_queue.cc:260] T f66c54d560644233a330b92df87046c6 P 599840b07db24ad088880d0fdc1c5cba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2c00b8f050b47fabe1abc7847e2db48" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 40101 } } peers { permanent_uuid: "2e01c23563984d0682838b27189624fa" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 42565 } } peers { permanent_uuid: "599840b07db24ad088880d0fdc1c5cba" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 44967 } }
I20260805 23:10:47.700266 1688 raft_consensus.cc:348] T f66c54d560644233a330b92df87046c6 P b2c00b8f050b47fabe1abc7847e2db48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2c00b8f050b47fabe1abc7847e2db48" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 40101 } } peers { permanent_uuid: "2e01c23563984d0682838b27189624fa" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 42565 } } peers { permanent_uuid: "599840b07db24ad088880d0fdc1c5cba" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 44967 } }
I20260805 23:10:47.700569 1688 raft_consensus.cc:374] T f66c54d560644233a330b92df87046c6 P b2c00b8f050b47fabe1abc7847e2db48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:10:47.700629 1514 heartbeater.cc:503] Master 127.25.92.254:38875 was elected leader, sending a full tablet report...
I20260805 23:10:47.700634 1688 raft_consensus.cc:729] T f66c54d560644233a330b92df87046c6 P b2c00b8f050b47fabe1abc7847e2db48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2c00b8f050b47fabe1abc7847e2db48, State: Initialized, Role: FOLLOWER
I20260805 23:10:47.700927 1687 ts_tablet_manager.cc:1428] T f66c54d560644233a330b92df87046c6 P 599840b07db24ad088880d0fdc1c5cba: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260805 23:10:47.701128 1688 consensus_queue.cc:260] T f66c54d560644233a330b92df87046c6 P b2c00b8f050b47fabe1abc7847e2db48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2c00b8f050b47fabe1abc7847e2db48" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 40101 } } peers { permanent_uuid: "2e01c23563984d0682838b27189624fa" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 42565 } } peers { permanent_uuid: "599840b07db24ad088880d0fdc1c5cba" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 44967 } }
I20260805 23:10:47.702250 1658 heartbeater.cc:503] Master 127.25.92.254:38875 was elected leader, sending a full tablet report...
I20260805 23:10:47.702643 1688 ts_tablet_manager.cc:1428] T f66c54d560644233a330b92df87046c6 P b2c00b8f050b47fabe1abc7847e2db48: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260805 23:10:47.842828 1692 raft_consensus.cc:482] T f66c54d560644233a330b92df87046c6 P 2e01c23563984d0682838b27189624fa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 23:10:47.843122 1692 raft_consensus.cc:504] T f66c54d560644233a330b92df87046c6 P 2e01c23563984d0682838b27189624fa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2c00b8f050b47fabe1abc7847e2db48" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 40101 } } peers { permanent_uuid: "2e01c23563984d0682838b27189624fa" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 42565 } } peers { permanent_uuid: "599840b07db24ad088880d0fdc1c5cba" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 44967 } }
I20260805 23:10:47.844393 1692 leader_election.cc:288] T f66c54d560644233a330b92df87046c6 P 2e01c23563984d0682838b27189624fa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b2c00b8f050b47fabe1abc7847e2db48 (127.25.92.195:40101), 599840b07db24ad088880d0fdc1c5cba (127.25.92.194:44967)
I20260805 23:10:47.849376 1685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:47.845658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.193:39373 (local address 127.25.92.194:44967)
0805 23:10:47.845857 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:47.845863 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:47.845975 (+ 112us) server_negotiation.cc:407] Connection header received
0805 23:10:47.846081 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:47.846085 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:47.846140 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:47.846231 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:10:47.846501 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:47.847071 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:47.848328 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:47.848770 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:47.848908 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:47.848965 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:47.849166 (+ 201us) server_negotiation.cc:299] Negotiation successful
0805 23:10:47.849247 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260805 23:10:47.849406 1696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:47.845401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.194:44967 (local address 127.25.92.193:39373)
0805 23:10:47.845869 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:47.845881 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:47.845981 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:47.846248 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:47.846250 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:47.846263 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:10:47.846371 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:47.846376 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:47.847202 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:47.847209 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:47.848179 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:47.848189 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:47.848902 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:47.848906 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:47.849024 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:47.849046 (+ 22us) client_negotiation.cc:769] Sending connection context
0805 23:10:47.849147 (+ 101us) client_negotiation.cc:240] Negotiation successful
0805 23:10:47.849224 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":394,"mutex_wait_us":25,"thread_start_us":148,"threads_started":1}
I20260805 23:10:47.849613 1682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:47.844955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.193:41981 (local address 127.25.92.195:40101)
0805 23:10:47.845107 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:47.845111 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:47.845375 (+ 264us) server_negotiation.cc:407] Connection header received
0805 23:10:47.845660 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:47.845665 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:47.845737 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:47.845824 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:10:47.846442 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:47.847039 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:47.848423 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:47.848906 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:47.849074 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:47.849142 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:47.849362 (+ 220us) server_negotiation.cc:299] Negotiation successful
0805 23:10:47.849447 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260805 23:10:47.849910 1695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:47.844819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.195:40101 (local address 127.25.92.193:41981)
0805 23:10:47.845245 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:47.845274 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:47.845433 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:47.846037 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:47.846040 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:47.846077 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:10:47.846241 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:47.846252 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:47.847181 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:47.847189 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:47.848259 (+ 1070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:47.848271 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:47.849060 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:47.849065 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:47.849196 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:47.849234 (+ 38us) client_negotiation.cc:769] Sending connection context
0805 23:10:47.849342 (+ 108us) client_negotiation.cc:240] Negotiation successful
0805 23:10:47.849437 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"mutex_wait_us":111,"thread_start_us":189,"threads_started":1}
I20260805 23:10:47.850454 1466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f66c54d560644233a330b92df87046c6" candidate_uuid: "2e01c23563984d0682838b27189624fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "599840b07db24ad088880d0fdc1c5cba" is_pre_election: true
I20260805 23:10:47.850647 1610 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f66c54d560644233a330b92df87046c6" candidate_uuid: "2e01c23563984d0682838b27189624fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2c00b8f050b47fabe1abc7847e2db48" is_pre_election: true
I20260805 23:10:47.850816 1466 raft_consensus.cc:2454] T f66c54d560644233a330b92df87046c6 P 599840b07db24ad088880d0fdc1c5cba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2e01c23563984d0682838b27189624fa in term 0.
I20260805 23:10:47.850994 1610 raft_consensus.cc:2454] T f66c54d560644233a330b92df87046c6 P b2c00b8f050b47fabe1abc7847e2db48 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2e01c23563984d0682838b27189624fa in term 0.
I20260805 23:10:47.851408 1251 leader_election.cc:302] T f66c54d560644233a330b92df87046c6 P 2e01c23563984d0682838b27189624fa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e01c23563984d0682838b27189624fa, 599840b07db24ad088880d0fdc1c5cba; no voters:
I20260805 23:10:47.851774 1692 raft_consensus.cc:2781] T f66c54d560644233a330b92df87046c6 P 2e01c23563984d0682838b27189624fa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260805 23:10:47.851861 1692 raft_consensus.cc:482] T f66c54d560644233a330b92df87046c6 P 2e01c23563984d0682838b27189624fa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260805 23:10:47.851917 1692 raft_consensus.cc:3037] T f66c54d560644233a330b92df87046c6 P 2e01c23563984d0682838b27189624fa [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:10:47.852973 1692 raft_consensus.cc:504] T f66c54d560644233a330b92df87046c6 P 2e01c23563984d0682838b27189624fa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2c00b8f050b47fabe1abc7847e2db48" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 40101 } } peers { permanent_uuid: "2e01c23563984d0682838b27189624fa" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 42565 } } peers { permanent_uuid: "599840b07db24ad088880d0fdc1c5cba" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 44967 } }
I20260805 23:10:47.853328 1692 leader_election.cc:288] T f66c54d560644233a330b92df87046c6 P 2e01c23563984d0682838b27189624fa [CANDIDATE]: Term 1 election: Requested vote from peers b2c00b8f050b47fabe1abc7847e2db48 (127.25.92.195:40101), 599840b07db24ad088880d0fdc1c5cba (127.25.92.194:44967)
I20260805 23:10:47.853832 1610 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f66c54d560644233a330b92df87046c6" candidate_uuid: "2e01c23563984d0682838b27189624fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2c00b8f050b47fabe1abc7847e2db48"
I20260805 23:10:47.853833 1466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f66c54d560644233a330b92df87046c6" candidate_uuid: "2e01c23563984d0682838b27189624fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "599840b07db24ad088880d0fdc1c5cba"
I20260805 23:10:47.853981 1466 raft_consensus.cc:3037] T f66c54d560644233a330b92df87046c6 P 599840b07db24ad088880d0fdc1c5cba [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:10:47.853981 1610 raft_consensus.cc:3037] T f66c54d560644233a330b92df87046c6 P b2c00b8f050b47fabe1abc7847e2db48 [term 0 FOLLOWER]: Advancing to term 1
W20260805 23:10:47.854136 1659 tablet.cc:2369] T f66c54d560644233a330b92df87046c6 P b2c00b8f050b47fabe1abc7847e2db48: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260805 23:10:47.855487 1466 raft_consensus.cc:2454] T f66c54d560644233a330b92df87046c6 P 599840b07db24ad088880d0fdc1c5cba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e01c23563984d0682838b27189624fa in term 1.
I20260805 23:10:47.855994 1251 leader_election.cc:302] T f66c54d560644233a330b92df87046c6 P 2e01c23563984d0682838b27189624fa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e01c23563984d0682838b27189624fa, 599840b07db24ad088880d0fdc1c5cba; no voters:
I20260805 23:10:47.856031 1610 raft_consensus.cc:2454] T f66c54d560644233a330b92df87046c6 P b2c00b8f050b47fabe1abc7847e2db48 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e01c23563984d0682838b27189624fa in term 1.
I20260805 23:10:47.856210 1692 raft_consensus.cc:2781] T f66c54d560644233a330b92df87046c6 P 2e01c23563984d0682838b27189624fa [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:10:47.856475 1692 raft_consensus.cc:686] T f66c54d560644233a330b92df87046c6 P 2e01c23563984d0682838b27189624fa [term 1 LEADER]: Becoming Leader. State: Replica: 2e01c23563984d0682838b27189624fa, State: Running, Role: LEADER
I20260805 23:10:47.856843 1692 consensus_queue.cc:237] T f66c54d560644233a330b92df87046c6 P 2e01c23563984d0682838b27189624fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2c00b8f050b47fabe1abc7847e2db48" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 40101 } } peers { permanent_uuid: "2e01c23563984d0682838b27189624fa" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 42565 } } peers { permanent_uuid: "599840b07db24ad088880d0fdc1c5cba" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 44967 } }
I20260805 23:10:47.860587 1158 catalog_manager.cc:5495] T f66c54d560644233a330b92df87046c6 P 2e01c23563984d0682838b27189624fa reported cstate change: term changed from 0 to 1, leader changed from <none> to 2e01c23563984d0682838b27189624fa (127.25.92.193). New cstate: current_term: 1 leader_uuid: "2e01c23563984d0682838b27189624fa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2c00b8f050b47fabe1abc7847e2db48" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 40101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2e01c23563984d0682838b27189624fa" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 42565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "599840b07db24ad088880d0fdc1c5cba" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 44967 } health_report { overall_health: UNKNOWN } } }
W20260805 23:10:47.881194 1515 tablet.cc:2369] T f66c54d560644233a330b92df87046c6 P 599840b07db24ad088880d0fdc1c5cba: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260805 23:10:47.885133 1371 tablet.cc:2369] T f66c54d560644233a330b92df87046c6 P 2e01c23563984d0682838b27189624fa: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260805 23:10:47.912400 1684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:47.908221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47532 (local address 127.25.92.193:42565)
0805 23:10:47.908355 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:47.908359 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:47.908373 (+ 14us) server_negotiation.cc:407] Connection header received
0805 23:10:47.908487 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:47.908489 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:47.908545 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:47.908628 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 23:10:47.908932 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:47.909431 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:47.910466 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:47.910750 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:47.910878 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:47.911323 (+ 445us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 23:10:47.911979 (+ 656us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 23:10:47.912079 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:47.912216 (+ 137us) server_negotiation.cc:299] Negotiation successful
0805 23:10:47.912277 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260805 23:10:47.922391 1466 raft_consensus.cc:1261] T f66c54d560644233a330b92df87046c6 P 599840b07db24ad088880d0fdc1c5cba [term 1 FOLLOWER]: Refusing update from remote peer 2e01c23563984d0682838b27189624fa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:10:47.922391 1610 raft_consensus.cc:1261] T f66c54d560644233a330b92df87046c6 P b2c00b8f050b47fabe1abc7847e2db48 [term 1 FOLLOWER]: Refusing update from remote peer 2e01c23563984d0682838b27189624fa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:10:47.923408 1698 consensus_queue.cc:1035] T f66c54d560644233a330b92df87046c6 P 2e01c23563984d0682838b27189624fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2c00b8f050b47fabe1abc7847e2db48" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 40101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:10:47.923745 1692 consensus_queue.cc:1035] T f66c54d560644233a330b92df87046c6 P 2e01c23563984d0682838b27189624fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "599840b07db24ad088880d0fdc1c5cba" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 44967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:10:47.932150 1701 mvcc.cc:205] Tried to move back new op lower bound from 7315339050679013376 to 7315339050424999936. Current Snapshot: MvccSnapshot[applied={T|T < 7315339050679013376}]
I20260805 23:10:47.933624 1703 mvcc.cc:205] Tried to move back new op lower bound from 7315339050679013376 to 7315339050424999936. Current Snapshot: MvccSnapshot[applied={T|T < 7315339050679013376}]
I20260805 23:10:47.934595 1702 mvcc.cc:205] Tried to move back new op lower bound from 7315339050679013376 to 7315339050424999936. Current Snapshot: MvccSnapshot[applied={T|T < 7315339050679013376}]
I20260805 23:10:47.942428 1162 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44170:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:10:47.942622 1162 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44170:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:10:47.945590 1162 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3d3f991025e246978b016ce9b762cfa0: Sending DeleteTablet for 3 replicas of tablet f66c54d560644233a330b92df87046c6
I20260805 23:10:47.946559 1446 tablet_service.cc:1514] Processing DeleteTablet for tablet f66c54d560644233a330b92df87046c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:10:47 UTC) from {username='oryx', principal='oryx/127.25.92.254@KRBTEST.COM'} at 127.0.0.1:60674
I20260805 23:10:47.946583 1303 tablet_service.cc:1514] Processing DeleteTablet for tablet f66c54d560644233a330b92df87046c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:10:47 UTC) from {username='oryx', principal='oryx/127.25.92.254@KRBTEST.COM'} at 127.0.0.1:47526
I20260805 23:10:47.946949 1590 tablet_service.cc:1514] Processing DeleteTablet for tablet f66c54d560644233a330b92df87046c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:10:47 UTC) from {username='oryx', principal='oryx/127.25.92.254@KRBTEST.COM'} at 127.0.0.1:34862
I20260805 23:10:47.947088 1711 tablet_replica.cc:330] T f66c54d560644233a330b92df87046c6 P 599840b07db24ad088880d0fdc1c5cba: stopping tablet replica
I20260805 23:10:47.947368 1712 tablet_replica.cc:330] T f66c54d560644233a330b92df87046c6 P 2e01c23563984d0682838b27189624fa: stopping tablet replica
I20260805 23:10:47.947607 1711 raft_consensus.cc:2229] T f66c54d560644233a330b92df87046c6 P 599840b07db24ad088880d0fdc1c5cba [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 23:10:47.947705 1712 raft_consensus.cc:2229] T f66c54d560644233a330b92df87046c6 P 2e01c23563984d0682838b27189624fa [term 1 LEADER]: Raft consensus shutting down.
I20260805 23:10:47.947772 1162 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44170:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260805 23:10:47.948083 1712 raft_consensus.cc:2258] T f66c54d560644233a330b92df87046c6 P 2e01c23563984d0682838b27189624fa [term 1 FOLLOWER]: Raft consensus is shut down!
W20260805 23:10:47.948233 1162 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260805 23:10:47.948393 1711 raft_consensus.cc:2258] T f66c54d560644233a330b92df87046c6 P 599840b07db24ad088880d0fdc1c5cba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:10:47.949676 1712 ts_tablet_manager.cc:1905] T f66c54d560644233a330b92df87046c6 P 2e01c23563984d0682838b27189624fa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:10:47.950011 1711 ts_tablet_manager.cc:1905] T f66c54d560644233a330b92df87046c6 P 599840b07db24ad088880d0fdc1c5cba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:10:47.952014 1713 tablet_replica.cc:330] T f66c54d560644233a330b92df87046c6 P b2c00b8f050b47fabe1abc7847e2db48: stopping tablet replica
I20260805 23:10:47.952364 1713 raft_consensus.cc:2229] T f66c54d560644233a330b92df87046c6 P b2c00b8f050b47fabe1abc7847e2db48 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 23:10:47.952703 1713 raft_consensus.cc:2258] T f66c54d560644233a330b92df87046c6 P b2c00b8f050b47fabe1abc7847e2db48 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:10:47.953152 1711 ts_tablet_manager.cc:1918] T f66c54d560644233a330b92df87046c6 P 599840b07db24ad088880d0fdc1c5cba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 23:10:47.953300 1711 log.cc:1196] T f66c54d560644233a330b92df87046c6 P 599840b07db24ad088880d0fdc1c5cba: Deleting WAL directory at /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/minicluster-data/ts-1/wal/wals/f66c54d560644233a330b92df87046c6
I20260805 23:10:47.953714 1711 ts_tablet_manager.cc:1939] T f66c54d560644233a330b92df87046c6 P 599840b07db24ad088880d0fdc1c5cba: Deleting consensus metadata
I20260805 23:10:47.954200 1303 tablet_service.cc:1467] Processing CreateTablet for tablet 51693ca5017442ba8da4bda95255cd59 (DEFAULT_TABLE table=test-table [id=4c4b138d75a04d3a9f3baa0d14754977]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:10:47.954478 1303 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 51693ca5017442ba8da4bda95255cd59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:10:47.954543 1713 ts_tablet_manager.cc:1905] T f66c54d560644233a330b92df87046c6 P b2c00b8f050b47fabe1abc7847e2db48: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:10:47.957075 1713 ts_tablet_manager.cc:1918] T f66c54d560644233a330b92df87046c6 P b2c00b8f050b47fabe1abc7847e2db48: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 23:10:47.957175 1713 log.cc:1196] T f66c54d560644233a330b92df87046c6 P b2c00b8f050b47fabe1abc7847e2db48: Deleting WAL directory at /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/minicluster-data/ts-2/wal/wals/f66c54d560644233a330b92df87046c6
I20260805 23:10:47.957250 1446 tablet_service.cc:1467] Processing CreateTablet for tablet 51693ca5017442ba8da4bda95255cd59 (DEFAULT_TABLE table=test-table [id=4c4b138d75a04d3a9f3baa0d14754977]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:10:47.957517 1713 ts_tablet_manager.cc:1939] T f66c54d560644233a330b92df87046c6 P b2c00b8f050b47fabe1abc7847e2db48: Deleting consensus metadata
I20260805 23:10:47.957671 1686 tablet_bootstrap.cc:492] T 51693ca5017442ba8da4bda95255cd59 P 2e01c23563984d0682838b27189624fa: Bootstrap starting.
I20260805 23:10:47.957753 1590 tablet_service.cc:1467] Processing CreateTablet for tablet 51693ca5017442ba8da4bda95255cd59 (DEFAULT_TABLE table=test-table [id=4c4b138d75a04d3a9f3baa0d14754977]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:10:47.957965 1590 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 51693ca5017442ba8da4bda95255cd59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:10:47.958635 1686 tablet_bootstrap.cc:654] T 51693ca5017442ba8da4bda95255cd59 P 2e01c23563984d0682838b27189624fa: Neither blocks nor log segments found. Creating new log.
I20260805 23:10:47.959843 1686 tablet_bootstrap.cc:492] T 51693ca5017442ba8da4bda95255cd59 P 2e01c23563984d0682838b27189624fa: No bootstrap required, opened a new log
I20260805 23:10:47.959934 1686 ts_tablet_manager.cc:1397] T 51693ca5017442ba8da4bda95255cd59 P 2e01c23563984d0682838b27189624fa: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260805 23:10:47.960207 1688 tablet_bootstrap.cc:492] T 51693ca5017442ba8da4bda95255cd59 P b2c00b8f050b47fabe1abc7847e2db48: Bootstrap starting.
I20260805 23:10:47.960404 1686 raft_consensus.cc:348] T 51693ca5017442ba8da4bda95255cd59 P 2e01c23563984d0682838b27189624fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e01c23563984d0682838b27189624fa" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 42565 } } peers { permanent_uuid: "599840b07db24ad088880d0fdc1c5cba" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 44967 } } peers { permanent_uuid: "b2c00b8f050b47fabe1abc7847e2db48" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 40101 } }
I20260805 23:10:47.960525 1686 raft_consensus.cc:374] T 51693ca5017442ba8da4bda95255cd59 P 2e01c23563984d0682838b27189624fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:10:47.960565 1686 raft_consensus.cc:729] T 51693ca5017442ba8da4bda95255cd59 P 2e01c23563984d0682838b27189624fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e01c23563984d0682838b27189624fa, State: Initialized, Role: FOLLOWER
I20260805 23:10:47.960723 1686 consensus_queue.cc:260] T 51693ca5017442ba8da4bda95255cd59 P 2e01c23563984d0682838b27189624fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e01c23563984d0682838b27189624fa" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 42565 } } peers { permanent_uuid: "599840b07db24ad088880d0fdc1c5cba" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 44967 } } peers { permanent_uuid: "b2c00b8f050b47fabe1abc7847e2db48" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 40101 } }
I20260805 23:10:47.961220 1688 tablet_bootstrap.cc:654] T 51693ca5017442ba8da4bda95255cd59 P b2c00b8f050b47fabe1abc7847e2db48: Neither blocks nor log segments found. Creating new log.
I20260805 23:10:47.961746 1144 catalog_manager.cc:4841] TS b2c00b8f050b47fabe1abc7847e2db48 (127.25.92.195:40101): tablet f66c54d560644233a330b92df87046c6 (table test-table [id=10ef581f78844329b91c97882ba14453]) successfully deleted
I20260805 23:10:47.961871 1686 ts_tablet_manager.cc:1428] T 51693ca5017442ba8da4bda95255cd59 P 2e01c23563984d0682838b27189624fa: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260805 23:10:47.962033 1144 catalog_manager.cc:4841] TS 599840b07db24ad088880d0fdc1c5cba (127.25.92.194:44967): tablet f66c54d560644233a330b92df87046c6 (table test-table [id=10ef581f78844329b91c97882ba14453]) successfully deleted
I20260805 23:10:47.962410 1446 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 51693ca5017442ba8da4bda95255cd59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:10:47.962615 1712 ts_tablet_manager.cc:1918] T f66c54d560644233a330b92df87046c6 P 2e01c23563984d0682838b27189624fa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 23:10:47.962765 1712 log.cc:1196] T f66c54d560644233a330b92df87046c6 P 2e01c23563984d0682838b27189624fa: Deleting WAL directory at /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/minicluster-data/ts-0/wal/wals/f66c54d560644233a330b92df87046c6
I20260805 23:10:47.963135 1712 ts_tablet_manager.cc:1939] T f66c54d560644233a330b92df87046c6 P 2e01c23563984d0682838b27189624fa: Deleting consensus metadata
I20260805 23:10:47.964386 1147 catalog_manager.cc:4841] TS 2e01c23563984d0682838b27189624fa (127.25.92.193:42565): tablet f66c54d560644233a330b92df87046c6 (table test-table [id=10ef581f78844329b91c97882ba14453]) successfully deleted
I20260805 23:10:47.964962 1687 tablet_bootstrap.cc:492] T 51693ca5017442ba8da4bda95255cd59 P 599840b07db24ad088880d0fdc1c5cba: Bootstrap starting.
I20260805 23:10:47.966051 1687 tablet_bootstrap.cc:654] T 51693ca5017442ba8da4bda95255cd59 P 599840b07db24ad088880d0fdc1c5cba: Neither blocks nor log segments found. Creating new log.
I20260805 23:10:47.968035 1687 tablet_bootstrap.cc:492] T 51693ca5017442ba8da4bda95255cd59 P 599840b07db24ad088880d0fdc1c5cba: No bootstrap required, opened a new log
I20260805 23:10:47.968115 1687 ts_tablet_manager.cc:1397] T 51693ca5017442ba8da4bda95255cd59 P 599840b07db24ad088880d0fdc1c5cba: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260805 23:10:47.968180 1688 tablet_bootstrap.cc:492] T 51693ca5017442ba8da4bda95255cd59 P b2c00b8f050b47fabe1abc7847e2db48: No bootstrap required, opened a new log
I20260805 23:10:47.968273 1688 ts_tablet_manager.cc:1397] T 51693ca5017442ba8da4bda95255cd59 P b2c00b8f050b47fabe1abc7847e2db48: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20260805 23:10:47.968461 1687 raft_consensus.cc:348] T 51693ca5017442ba8da4bda95255cd59 P 599840b07db24ad088880d0fdc1c5cba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e01c23563984d0682838b27189624fa" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 42565 } } peers { permanent_uuid: "599840b07db24ad088880d0fdc1c5cba" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 44967 } } peers { permanent_uuid: "b2c00b8f050b47fabe1abc7847e2db48" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 40101 } }
I20260805 23:10:47.968555 1687 raft_consensus.cc:374] T 51693ca5017442ba8da4bda95255cd59 P 599840b07db24ad088880d0fdc1c5cba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:10:47.968577 1687 raft_consensus.cc:729] T 51693ca5017442ba8da4bda95255cd59 P 599840b07db24ad088880d0fdc1c5cba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 599840b07db24ad088880d0fdc1c5cba, State: Initialized, Role: FOLLOWER
I20260805 23:10:47.968714 1687 consensus_queue.cc:260] T 51693ca5017442ba8da4bda95255cd59 P 599840b07db24ad088880d0fdc1c5cba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e01c23563984d0682838b27189624fa" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 42565 } } peers { permanent_uuid: "599840b07db24ad088880d0fdc1c5cba" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 44967 } } peers { permanent_uuid: "b2c00b8f050b47fabe1abc7847e2db48" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 40101 } }
I20260805 23:10:47.968736 1688 raft_consensus.cc:348] T 51693ca5017442ba8da4bda95255cd59 P b2c00b8f050b47fabe1abc7847e2db48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e01c23563984d0682838b27189624fa" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 42565 } } peers { permanent_uuid: "599840b07db24ad088880d0fdc1c5cba" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 44967 } } peers { permanent_uuid: "b2c00b8f050b47fabe1abc7847e2db48" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 40101 } }
I20260805 23:10:47.968849 1688 raft_consensus.cc:374] T 51693ca5017442ba8da4bda95255cd59 P b2c00b8f050b47fabe1abc7847e2db48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:10:47.968878 1688 raft_consensus.cc:729] T 51693ca5017442ba8da4bda95255cd59 P b2c00b8f050b47fabe1abc7847e2db48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2c00b8f050b47fabe1abc7847e2db48, State: Initialized, Role: FOLLOWER
I20260805 23:10:47.968990 1687 ts_tablet_manager.cc:1428] T 51693ca5017442ba8da4bda95255cd59 P 599840b07db24ad088880d0fdc1c5cba: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260805 23:10:47.969000 1688 consensus_queue.cc:260] T 51693ca5017442ba8da4bda95255cd59 P b2c00b8f050b47fabe1abc7847e2db48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e01c23563984d0682838b27189624fa" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 42565 } } peers { permanent_uuid: "599840b07db24ad088880d0fdc1c5cba" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 44967 } } peers { permanent_uuid: "b2c00b8f050b47fabe1abc7847e2db48" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 40101 } }
I20260805 23:10:47.969240 1688 ts_tablet_manager.cc:1428] T 51693ca5017442ba8da4bda95255cd59 P b2c00b8f050b47fabe1abc7847e2db48: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260805 23:10:48.141085 1694 raft_consensus.cc:482] T 51693ca5017442ba8da4bda95255cd59 P b2c00b8f050b47fabe1abc7847e2db48 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 23:10:48.141280 1694 raft_consensus.cc:504] T 51693ca5017442ba8da4bda95255cd59 P b2c00b8f050b47fabe1abc7847e2db48 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e01c23563984d0682838b27189624fa" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 42565 } } peers { permanent_uuid: "599840b07db24ad088880d0fdc1c5cba" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 44967 } } peers { permanent_uuid: "b2c00b8f050b47fabe1abc7847e2db48" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 40101 } }
I20260805 23:10:48.142498 1694 leader_election.cc:288] T 51693ca5017442ba8da4bda95255cd59 P b2c00b8f050b47fabe1abc7847e2db48 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2e01c23563984d0682838b27189624fa (127.25.92.193:42565), 599840b07db24ad088880d0fdc1c5cba (127.25.92.194:44967)
I20260805 23:10:48.146539 1717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:48.142739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.193:42565 (local address 127.25.92.195:35317)
0805 23:10:48.143006 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:48.143022 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:48.143191 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:48.143617 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:48.143620 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:48.143644 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:10:48.143747 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:48.143752 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:48.144501 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:48.144506 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:48.145350 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:48.145371 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:48.146067 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:48.146071 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:48.146181 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:48.146211 (+ 30us) client_negotiation.cc:769] Sending connection context
0805 23:10:48.146301 (+ 90us) client_negotiation.cc:240] Negotiation successful
0805 23:10:48.146382 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":78,"threads_started":1}
I20260805 23:10:48.146520 1684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:48.142863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.195:35317 (local address 127.25.92.193:42565)
0805 23:10:48.143012 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:48.143019 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:48.143105 (+ 86us) server_negotiation.cc:407] Connection header received
0805 23:10:48.143333 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:48.143338 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:48.143402 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:48.143455 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:10:48.143869 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:48.144381 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:48.145537 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:48.145938 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:48.146075 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:48.146135 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:48.146320 (+ 185us) server_negotiation.cc:299] Negotiation successful
0805 23:10:48.146401 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260805 23:10:48.147257 1323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51693ca5017442ba8da4bda95255cd59" candidate_uuid: "b2c00b8f050b47fabe1abc7847e2db48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e01c23563984d0682838b27189624fa" is_pre_election: true
I20260805 23:10:48.147495 1685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:48.143144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.195:48627 (local address 127.25.92.194:44967)
0805 23:10:48.143287 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:48.143294 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:48.143309 (+ 15us) server_negotiation.cc:407] Connection header received
0805 23:10:48.143429 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:48.143435 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:48.143483 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:48.143559 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:10:48.143896 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:48.144680 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:48.146158 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:48.146787 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:48.146876 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:48.146921 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:48.147279 (+ 358us) server_negotiation.cc:299] Negotiation successful
0805 23:10:48.147363 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260805 23:10:48.147516 1323 raft_consensus.cc:2454] T 51693ca5017442ba8da4bda95255cd59 P 2e01c23563984d0682838b27189624fa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2c00b8f050b47fabe1abc7847e2db48 in term 0.
I20260805 23:10:48.147668 1718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:48.142802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.194:44967 (local address 127.25.92.195:48627)
0805 23:10:48.143211 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:48.143224 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:48.143314 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:48.143600 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:48.143603 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:48.143621 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:10:48.143761 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:48.143767 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:48.144816 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:48.144819 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:48.146014 (+ 1195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:48.146024 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:48.147033 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:48.147036 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:48.147149 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:48.147170 (+ 21us) client_negotiation.cc:769] Sending connection context
0805 23:10:48.147236 (+ 66us) client_negotiation.cc:240] Negotiation successful
0805 23:10:48.147303 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":347,"spinlock_wait_cycles":9472,"thread_start_us":88,"threads_started":1}
I20260805 23:10:48.148000 1542 leader_election.cc:302] T 51693ca5017442ba8da4bda95255cd59 P b2c00b8f050b47fabe1abc7847e2db48 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e01c23563984d0682838b27189624fa, b2c00b8f050b47fabe1abc7847e2db48; no voters:
I20260805 23:10:48.148336 1694 raft_consensus.cc:2781] T 51693ca5017442ba8da4bda95255cd59 P b2c00b8f050b47fabe1abc7847e2db48 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260805 23:10:48.148402 1694 raft_consensus.cc:482] T 51693ca5017442ba8da4bda95255cd59 P b2c00b8f050b47fabe1abc7847e2db48 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260805 23:10:48.148438 1694 raft_consensus.cc:3037] T 51693ca5017442ba8da4bda95255cd59 P b2c00b8f050b47fabe1abc7847e2db48 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:10:48.148460 1466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51693ca5017442ba8da4bda95255cd59" candidate_uuid: "b2c00b8f050b47fabe1abc7847e2db48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "599840b07db24ad088880d0fdc1c5cba" is_pre_election: true
I20260805 23:10:48.148645 1466 raft_consensus.cc:2454] T 51693ca5017442ba8da4bda95255cd59 P 599840b07db24ad088880d0fdc1c5cba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2c00b8f050b47fabe1abc7847e2db48 in term 0.
I20260805 23:10:48.149699 1694 raft_consensus.cc:504] T 51693ca5017442ba8da4bda95255cd59 P b2c00b8f050b47fabe1abc7847e2db48 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e01c23563984d0682838b27189624fa" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 42565 } } peers { permanent_uuid: "599840b07db24ad088880d0fdc1c5cba" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 44967 } } peers { permanent_uuid: "b2c00b8f050b47fabe1abc7847e2db48" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 40101 } }
I20260805 23:10:48.150079 1694 leader_election.cc:288] T 51693ca5017442ba8da4bda95255cd59 P b2c00b8f050b47fabe1abc7847e2db48 [CANDIDATE]: Term 1 election: Requested vote from peers 2e01c23563984d0682838b27189624fa (127.25.92.193:42565), 599840b07db24ad088880d0fdc1c5cba (127.25.92.194:44967)
I20260805 23:10:48.150547 1323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51693ca5017442ba8da4bda95255cd59" candidate_uuid: "b2c00b8f050b47fabe1abc7847e2db48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e01c23563984d0682838b27189624fa"
I20260805 23:10:48.150584 1466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51693ca5017442ba8da4bda95255cd59" candidate_uuid: "b2c00b8f050b47fabe1abc7847e2db48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "599840b07db24ad088880d0fdc1c5cba"
I20260805 23:10:48.150707 1466 raft_consensus.cc:3037] T 51693ca5017442ba8da4bda95255cd59 P 599840b07db24ad088880d0fdc1c5cba [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:10:48.150714 1323 raft_consensus.cc:3037] T 51693ca5017442ba8da4bda95255cd59 P 2e01c23563984d0682838b27189624fa [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:10:48.151767 1466 raft_consensus.cc:2454] T 51693ca5017442ba8da4bda95255cd59 P 599840b07db24ad088880d0fdc1c5cba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2c00b8f050b47fabe1abc7847e2db48 in term 1.
I20260805 23:10:48.151793 1323 raft_consensus.cc:2454] T 51693ca5017442ba8da4bda95255cd59 P 2e01c23563984d0682838b27189624fa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2c00b8f050b47fabe1abc7847e2db48 in term 1.
I20260805 23:10:48.152135 1542 leader_election.cc:302] T 51693ca5017442ba8da4bda95255cd59 P b2c00b8f050b47fabe1abc7847e2db48 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e01c23563984d0682838b27189624fa, b2c00b8f050b47fabe1abc7847e2db48; no voters:
I20260805 23:10:48.152331 1694 raft_consensus.cc:2781] T 51693ca5017442ba8da4bda95255cd59 P b2c00b8f050b47fabe1abc7847e2db48 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:10:48.152568 1694 raft_consensus.cc:686] T 51693ca5017442ba8da4bda95255cd59 P b2c00b8f050b47fabe1abc7847e2db48 [term 1 LEADER]: Becoming Leader. State: Replica: b2c00b8f050b47fabe1abc7847e2db48, State: Running, Role: LEADER
I20260805 23:10:48.152853 1694 consensus_queue.cc:237] T 51693ca5017442ba8da4bda95255cd59 P b2c00b8f050b47fabe1abc7847e2db48 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e01c23563984d0682838b27189624fa" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 42565 } } peers { permanent_uuid: "599840b07db24ad088880d0fdc1c5cba" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 44967 } } peers { permanent_uuid: "b2c00b8f050b47fabe1abc7847e2db48" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 40101 } }
I20260805 23:10:48.155405 1162 catalog_manager.cc:5495] T 51693ca5017442ba8da4bda95255cd59 P b2c00b8f050b47fabe1abc7847e2db48 reported cstate change: term changed from 0 to 1, leader changed from <none> to b2c00b8f050b47fabe1abc7847e2db48 (127.25.92.195). New cstate: current_term: 1 leader_uuid: "b2c00b8f050b47fabe1abc7847e2db48" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e01c23563984d0682838b27189624fa" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 42565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "599840b07db24ad088880d0fdc1c5cba" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 44967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2c00b8f050b47fabe1abc7847e2db48" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 40101 } health_report { overall_health: HEALTHY } } }
Aug 05 23:10:48 dist-test-slave-ss4n krb5kdc[1113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971446, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.92.254@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM
I20260805 23:10:48.230430 1726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:48.219342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:38875 (local address 127.0.0.1:44182)
0805 23:10:48.220031 (+ 689us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:48.220052 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:48.220223 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:48.220558 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:48.220561 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:48.221275 (+ 714us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:48.221421 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:48.221433 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:48.222657 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:48.222661 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:48.223683 (+ 1022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:48.223696 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:48.224185 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:48.224188 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:48.224258 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:48.224992 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:10:48.225006 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:48.226861 (+ 1855us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:10:48.228627 (+ 1766us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:48.228637 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:48.228644 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:48.228922 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:48.229207 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:48.229209 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:48.229210 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:48.229301 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:48.229824 (+ 523us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:10:48.229831 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:10:48.229962 (+ 131us) client_negotiation.cc:769] Sending connection context
0805 23:10:48.230096 (+ 134us) client_negotiation.cc:240] Negotiation successful
0805 23:10:48.230238 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":537,"thread_start_us":130,"threads_started":1}
I20260805 23:10:48.230448 1727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:48.219387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44182 (local address 127.25.92.254:38875)
0805 23:10:48.219649 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:48.219656 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:48.220165 (+ 509us) server_negotiation.cc:407] Connection header received
0805 23:10:48.220364 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:48.220375 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:48.220419 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:48.220527 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:48.221663 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:48.222508 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:48.223840 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:48.224072 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:48.224160 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:48.227016 (+ 2856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:48.227019 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:48.227021 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:48.227042 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:48.228509 (+ 1467us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:48.229032 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:48.229035 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:48.229036 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:48.229116 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:48.229385 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:48.229388 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:48.229389 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:48.229690 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:48.229773 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:48.230185 (+ 412us) server_negotiation.cc:299] Negotiation successful
0805 23:10:48.230299 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":107,"threads_started":1}
I20260805 23:10:48.235049 1162 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.25.92.254@KRBTEST.COM'} at 127.0.0.1:44182:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260805 23:10:48.235836 1162 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260805 23:10:48.241110 1303 tablet_service.cc:1467] Processing CreateTablet for tablet c8fabc3af751454c89f470d4c1e8f915 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=76e5ccb852fe46298846e265c4495c68]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260805 23:10:48.241221 1446 tablet_service.cc:1467] Processing CreateTablet for tablet c8fabc3af751454c89f470d4c1e8f915 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=76e5ccb852fe46298846e265c4495c68]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260805 23:10:48.241609 1303 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8fabc3af751454c89f470d4c1e8f915. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:10:48.241609 1446 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8fabc3af751454c89f470d4c1e8f915. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:10:48.244638 1686 tablet_bootstrap.cc:492] T c8fabc3af751454c89f470d4c1e8f915 P 2e01c23563984d0682838b27189624fa: Bootstrap starting.
I20260805 23:10:48.244673 1687 tablet_bootstrap.cc:492] T c8fabc3af751454c89f470d4c1e8f915 P 599840b07db24ad088880d0fdc1c5cba: Bootstrap starting.
I20260805 23:10:48.245054 1590 tablet_service.cc:1467] Processing CreateTablet for tablet c8fabc3af751454c89f470d4c1e8f915 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=76e5ccb852fe46298846e265c4495c68]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260805 23:10:48.245364 1590 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8fabc3af751454c89f470d4c1e8f915. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:10:48.245645 1686 tablet_bootstrap.cc:654] T c8fabc3af751454c89f470d4c1e8f915 P 2e01c23563984d0682838b27189624fa: Neither blocks nor log segments found. Creating new log.
I20260805 23:10:48.245656 1687 tablet_bootstrap.cc:654] T c8fabc3af751454c89f470d4c1e8f915 P 599840b07db24ad088880d0fdc1c5cba: Neither blocks nor log segments found. Creating new log.
I20260805 23:10:48.246877 1686 tablet_bootstrap.cc:492] T c8fabc3af751454c89f470d4c1e8f915 P 2e01c23563984d0682838b27189624fa: No bootstrap required, opened a new log
I20260805 23:10:48.246978 1686 ts_tablet_manager.cc:1397] T c8fabc3af751454c89f470d4c1e8f915 P 2e01c23563984d0682838b27189624fa: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260805 23:10:48.247197 1687 tablet_bootstrap.cc:492] T c8fabc3af751454c89f470d4c1e8f915 P 599840b07db24ad088880d0fdc1c5cba: No bootstrap required, opened a new log
I20260805 23:10:48.247287 1687 ts_tablet_manager.cc:1397] T c8fabc3af751454c89f470d4c1e8f915 P 599840b07db24ad088880d0fdc1c5cba: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260805 23:10:48.247452 1686 raft_consensus.cc:348] T c8fabc3af751454c89f470d4c1e8f915 P 2e01c23563984d0682838b27189624fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e01c23563984d0682838b27189624fa" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 42565 } } peers { permanent_uuid: "599840b07db24ad088880d0fdc1c5cba" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 44967 } } peers { permanent_uuid: "b2c00b8f050b47fabe1abc7847e2db48" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 40101 } }
I20260805 23:10:48.247575 1686 raft_consensus.cc:374] T c8fabc3af751454c89f470d4c1e8f915 P 2e01c23563984d0682838b27189624fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:10:48.247611 1686 raft_consensus.cc:729] T c8fabc3af751454c89f470d4c1e8f915 P 2e01c23563984d0682838b27189624fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e01c23563984d0682838b27189624fa, State: Initialized, Role: FOLLOWER
I20260805 23:10:48.247752 1686 consensus_queue.cc:260] T c8fabc3af751454c89f470d4c1e8f915 P 2e01c23563984d0682838b27189624fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e01c23563984d0682838b27189624fa" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 42565 } } peers { permanent_uuid: "599840b07db24ad088880d0fdc1c5cba" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 44967 } } peers { permanent_uuid: "b2c00b8f050b47fabe1abc7847e2db48" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 40101 } }
I20260805 23:10:48.247756 1687 raft_consensus.cc:348] T c8fabc3af751454c89f470d4c1e8f915 P 599840b07db24ad088880d0fdc1c5cba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e01c23563984d0682838b27189624fa" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 42565 } } peers { permanent_uuid: "599840b07db24ad088880d0fdc1c5cba" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 44967 } } peers { permanent_uuid: "b2c00b8f050b47fabe1abc7847e2db48" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 40101 } }
I20260805 23:10:48.247853 1687 raft_consensus.cc:374] T c8fabc3af751454c89f470d4c1e8f915 P 599840b07db24ad088880d0fdc1c5cba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:10:48.247896 1687 raft_consensus.cc:729] T c8fabc3af751454c89f470d4c1e8f915 P 599840b07db24ad088880d0fdc1c5cba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 599840b07db24ad088880d0fdc1c5cba, State: Initialized, Role: FOLLOWER
I20260805 23:10:48.248014 1687 consensus_queue.cc:260] T c8fabc3af751454c89f470d4c1e8f915 P 599840b07db24ad088880d0fdc1c5cba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e01c23563984d0682838b27189624fa" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 42565 } } peers { permanent_uuid: "599840b07db24ad088880d0fdc1c5cba" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 44967 } } peers { permanent_uuid: "b2c00b8f050b47fabe1abc7847e2db48" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 40101 } }
I20260805 23:10:48.248011 1698 tablet_replica.cc:439] T c8fabc3af751454c89f470d4c1e8f915 P 2e01c23563984d0682838b27189624fa: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e01c23563984d0682838b27189624fa" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 42565 } } peers { permanent_uuid: "599840b07db24ad088880d0fdc1c5cba" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 44967 } } peers { permanent_uuid: "b2c00b8f050b47fabe1abc7847e2db48" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 40101 } } }
I20260805 23:10:48.248136 1698 tablet_replica.cc:442] T c8fabc3af751454c89f470d4c1e8f915 P 2e01c23563984d0682838b27189624fa: This TxnStatusTablet replica's current role is: FOLLOWER
I20260805 23:10:48.248298 1687 ts_tablet_manager.cc:1428] T c8fabc3af751454c89f470d4c1e8f915 P 599840b07db24ad088880d0fdc1c5cba: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260805 23:10:48.248382 1688 tablet_bootstrap.cc:492] T c8fabc3af751454c89f470d4c1e8f915 P b2c00b8f050b47fabe1abc7847e2db48: Bootstrap starting.
I20260805 23:10:48.248493 1693 tablet_replica.cc:439] T c8fabc3af751454c89f470d4c1e8f915 P 599840b07db24ad088880d0fdc1c5cba: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e01c23563984d0682838b27189624fa" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 42565 } } peers { permanent_uuid: "599840b07db24ad088880d0fdc1c5cba" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 44967 } } peers { permanent_uuid: "b2c00b8f050b47fabe1abc7847e2db48" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 40101 } } }
I20260805 23:10:48.248627 1693 tablet_replica.cc:442] T c8fabc3af751454c89f470d4c1e8f915 P 599840b07db24ad088880d0fdc1c5cba: This TxnStatusTablet replica's current role is: FOLLOWER
I20260805 23:10:48.249346 1688 tablet_bootstrap.cc:654] T c8fabc3af751454c89f470d4c1e8f915 P b2c00b8f050b47fabe1abc7847e2db48: Neither blocks nor log segments found. Creating new log.
I20260805 23:10:48.250541 1686 ts_tablet_manager.cc:1428] T c8fabc3af751454c89f470d4c1e8f915 P 2e01c23563984d0682838b27189624fa: Time spent starting tablet: real 0.004s user 0.000s sys 0.001s
I20260805 23:10:48.251106 1688 tablet_bootstrap.cc:492] T c8fabc3af751454c89f470d4c1e8f915 P b2c00b8f050b47fabe1abc7847e2db48: No bootstrap required, opened a new log
I20260805 23:10:48.251230 1688 ts_tablet_manager.cc:1397] T c8fabc3af751454c89f470d4c1e8f915 P b2c00b8f050b47fabe1abc7847e2db48: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260805 23:10:48.251631 1688 raft_consensus.cc:348] T c8fabc3af751454c89f470d4c1e8f915 P b2c00b8f050b47fabe1abc7847e2db48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e01c23563984d0682838b27189624fa" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 42565 } } peers { permanent_uuid: "599840b07db24ad088880d0fdc1c5cba" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 44967 } } peers { permanent_uuid: "b2c00b8f050b47fabe1abc7847e2db48" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 40101 } }
I20260805 23:10:48.251754 1688 raft_consensus.cc:374] T c8fabc3af751454c89f470d4c1e8f915 P b2c00b8f050b47fabe1abc7847e2db48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:10:48.251796 1688 raft_consensus.cc:729] T c8fabc3af751454c89f470d4c1e8f915 P b2c00b8f050b47fabe1abc7847e2db48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2c00b8f050b47fabe1abc7847e2db48, State: Initialized, Role: FOLLOWER
I20260805 23:10:48.251942 1688 consensus_queue.cc:260] T c8fabc3af751454c89f470d4c1e8f915 P b2c00b8f050b47fabe1abc7847e2db48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e01c23563984d0682838b27189624fa" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 42565 } } peers { permanent_uuid: "599840b07db24ad088880d0fdc1c5cba" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 44967 } } peers { permanent_uuid: "b2c00b8f050b47fabe1abc7847e2db48" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 40101 } }
I20260805 23:10:48.252238 1688 ts_tablet_manager.cc:1428] T c8fabc3af751454c89f470d4c1e8f915 P b2c00b8f050b47fabe1abc7847e2db48: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260805 23:10:48.252178 1694 tablet_replica.cc:439] T c8fabc3af751454c89f470d4c1e8f915 P b2c00b8f050b47fabe1abc7847e2db48: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e01c23563984d0682838b27189624fa" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 42565 } } peers { permanent_uuid: "599840b07db24ad088880d0fdc1c5cba" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 44967 } } peers { permanent_uuid: "b2c00b8f050b47fabe1abc7847e2db48" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 40101 } } }
I20260805 23:10:48.252329 1694 tablet_replica.cc:442] T c8fabc3af751454c89f470d4c1e8f915 P b2c00b8f050b47fabe1abc7847e2db48: This TxnStatusTablet replica's current role is: FOLLOWER
I20260805 23:10:48.294165 1698 raft_consensus.cc:482] T c8fabc3af751454c89f470d4c1e8f915 P 2e01c23563984d0682838b27189624fa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 23:10:48.294318 1698 raft_consensus.cc:504] T c8fabc3af751454c89f470d4c1e8f915 P 2e01c23563984d0682838b27189624fa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e01c23563984d0682838b27189624fa" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 42565 } } peers { permanent_uuid: "599840b07db24ad088880d0fdc1c5cba" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 44967 } } peers { permanent_uuid: "b2c00b8f050b47fabe1abc7847e2db48" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 40101 } }
I20260805 23:10:48.294704 1698 leader_election.cc:288] T c8fabc3af751454c89f470d4c1e8f915 P 2e01c23563984d0682838b27189624fa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 599840b07db24ad088880d0fdc1c5cba (127.25.92.194:44967), b2c00b8f050b47fabe1abc7847e2db48 (127.25.92.195:40101)
I20260805 23:10:48.295320 1466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8fabc3af751454c89f470d4c1e8f915" candidate_uuid: "2e01c23563984d0682838b27189624fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "599840b07db24ad088880d0fdc1c5cba" is_pre_election: true
I20260805 23:10:48.295362 1610 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8fabc3af751454c89f470d4c1e8f915" candidate_uuid: "2e01c23563984d0682838b27189624fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2c00b8f050b47fabe1abc7847e2db48" is_pre_election: true
I20260805 23:10:48.295514 1610 raft_consensus.cc:2454] T c8fabc3af751454c89f470d4c1e8f915 P b2c00b8f050b47fabe1abc7847e2db48 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2e01c23563984d0682838b27189624fa in term 0.
I20260805 23:10:48.295508 1466 raft_consensus.cc:2454] T c8fabc3af751454c89f470d4c1e8f915 P 599840b07db24ad088880d0fdc1c5cba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2e01c23563984d0682838b27189624fa in term 0.
I20260805 23:10:48.295874 1251 leader_election.cc:302] T c8fabc3af751454c89f470d4c1e8f915 P 2e01c23563984d0682838b27189624fa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e01c23563984d0682838b27189624fa, 599840b07db24ad088880d0fdc1c5cba; no voters:
I20260805 23:10:48.296128 1698 raft_consensus.cc:2781] T c8fabc3af751454c89f470d4c1e8f915 P 2e01c23563984d0682838b27189624fa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260805 23:10:48.296200 1698 raft_consensus.cc:482] T c8fabc3af751454c89f470d4c1e8f915 P 2e01c23563984d0682838b27189624fa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260805 23:10:48.296260 1698 raft_consensus.cc:3037] T c8fabc3af751454c89f470d4c1e8f915 P 2e01c23563984d0682838b27189624fa [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:10:48.297674 1698 raft_consensus.cc:504] T c8fabc3af751454c89f470d4c1e8f915 P 2e01c23563984d0682838b27189624fa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e01c23563984d0682838b27189624fa" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 42565 } } peers { permanent_uuid: "599840b07db24ad088880d0fdc1c5cba" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 44967 } } peers { permanent_uuid: "b2c00b8f050b47fabe1abc7847e2db48" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 40101 } }
I20260805 23:10:48.298066 1698 leader_election.cc:288] T c8fabc3af751454c89f470d4c1e8f915 P 2e01c23563984d0682838b27189624fa [CANDIDATE]: Term 1 election: Requested vote from peers 599840b07db24ad088880d0fdc1c5cba (127.25.92.194:44967), b2c00b8f050b47fabe1abc7847e2db48 (127.25.92.195:40101)
I20260805 23:10:48.298506 1466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8fabc3af751454c89f470d4c1e8f915" candidate_uuid: "2e01c23563984d0682838b27189624fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "599840b07db24ad088880d0fdc1c5cba"
I20260805 23:10:48.298565 1610 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8fabc3af751454c89f470d4c1e8f915" candidate_uuid: "2e01c23563984d0682838b27189624fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2c00b8f050b47fabe1abc7847e2db48"
I20260805 23:10:48.298650 1466 raft_consensus.cc:3037] T c8fabc3af751454c89f470d4c1e8f915 P 599840b07db24ad088880d0fdc1c5cba [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:10:48.298709 1610 raft_consensus.cc:3037] T c8fabc3af751454c89f470d4c1e8f915 P b2c00b8f050b47fabe1abc7847e2db48 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:10:48.299595 1466 raft_consensus.cc:2454] T c8fabc3af751454c89f470d4c1e8f915 P 599840b07db24ad088880d0fdc1c5cba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e01c23563984d0682838b27189624fa in term 1.
I20260805 23:10:48.299603 1610 raft_consensus.cc:2454] T c8fabc3af751454c89f470d4c1e8f915 P b2c00b8f050b47fabe1abc7847e2db48 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e01c23563984d0682838b27189624fa in term 1.
I20260805 23:10:48.299917 1251 leader_election.cc:302] T c8fabc3af751454c89f470d4c1e8f915 P 2e01c23563984d0682838b27189624fa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e01c23563984d0682838b27189624fa, b2c00b8f050b47fabe1abc7847e2db48; no voters:
I20260805 23:10:48.300156 1698 raft_consensus.cc:2781] T c8fabc3af751454c89f470d4c1e8f915 P 2e01c23563984d0682838b27189624fa [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:10:48.300282 1698 raft_consensus.cc:686] T c8fabc3af751454c89f470d4c1e8f915 P 2e01c23563984d0682838b27189624fa [term 1 LEADER]: Becoming Leader. State: Replica: 2e01c23563984d0682838b27189624fa, State: Running, Role: LEADER
I20260805 23:10:48.300424 1698 consensus_queue.cc:237] T c8fabc3af751454c89f470d4c1e8f915 P 2e01c23563984d0682838b27189624fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e01c23563984d0682838b27189624fa" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 42565 } } peers { permanent_uuid: "599840b07db24ad088880d0fdc1c5cba" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 44967 } } peers { permanent_uuid: "b2c00b8f050b47fabe1abc7847e2db48" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 40101 } }
I20260805 23:10:48.301156 1707 tablet_replica.cc:439] T c8fabc3af751454c89f470d4c1e8f915 P 2e01c23563984d0682838b27189624fa: TxnStatusTablet state changed. Reason: New leader 2e01c23563984d0682838b27189624fa. Latest consensus state: current_term: 1 leader_uuid: "2e01c23563984d0682838b27189624fa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e01c23563984d0682838b27189624fa" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 42565 } } peers { permanent_uuid: "599840b07db24ad088880d0fdc1c5cba" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 44967 } } peers { permanent_uuid: "b2c00b8f050b47fabe1abc7847e2db48" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 40101 } } }
I20260805 23:10:48.301265 1707 tablet_replica.cc:442] T c8fabc3af751454c89f470d4c1e8f915 P 2e01c23563984d0682838b27189624fa: This TxnStatusTablet replica's current role is: LEADER
I20260805 23:10:48.301597 1732 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260805 23:10:48.301782 1732 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260805 23:10:48.302343 1162 catalog_manager.cc:5495] T c8fabc3af751454c89f470d4c1e8f915 P 2e01c23563984d0682838b27189624fa reported cstate change: term changed from 0 to 1, leader changed from <none> to 2e01c23563984d0682838b27189624fa (127.25.92.193). New cstate: current_term: 1 leader_uuid: "2e01c23563984d0682838b27189624fa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e01c23563984d0682838b27189624fa" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 42565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "599840b07db24ad088880d0fdc1c5cba" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 44967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2c00b8f050b47fabe1abc7847e2db48" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 40101 } health_report { overall_health: UNKNOWN } } }
I20260805 23:10:48.319979 1726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:48.316130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.193:42565 (local address 127.0.0.1:47536)
0805 23:10:48.316302 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:48.316320 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:48.316424 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:48.316770 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:48.316773 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:48.316781 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0805 23:10:48.316884 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:48.316889 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:48.317952 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:48.317955 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:48.318628 (+ 673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:48.318639 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:48.319224 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:48.319226 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:48.319304 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:48.319357 (+ 53us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0805 23:10:48.319668 (+ 311us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0805 23:10:48.319683 (+ 15us) client_negotiation.cc:769] Sending connection context
0805 23:10:48.319742 (+ 59us) client_negotiation.cc:240] Negotiation successful
0805 23:10:48.319823 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260805 23:10:48.319973 1684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:48.316267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47536 (local address 127.25.92.193:42565)
0805 23:10:48.316387 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:48.316392 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:48.316408 (+ 16us) server_negotiation.cc:407] Connection header received
0805 23:10:48.316550 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:48.316553 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:48.316609 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:48.316710 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 23:10:48.317004 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:48.317819 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:48.318755 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:48.319103 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:48.319210 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:48.319493 (+ 283us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 23:10:48.319579 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 23:10:48.319658 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:48.319775 (+ 117us) server_negotiation.cc:299] Negotiation successful
0805 23:10:48.319836 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260805 23:10:48.322962 1610 raft_consensus.cc:1261] T c8fabc3af751454c89f470d4c1e8f915 P b2c00b8f050b47fabe1abc7847e2db48 [term 1 FOLLOWER]: Refusing update from remote peer 2e01c23563984d0682838b27189624fa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:10:48.323026 1466 raft_consensus.cc:1261] T c8fabc3af751454c89f470d4c1e8f915 P 599840b07db24ad088880d0fdc1c5cba [term 1 FOLLOWER]: Refusing update from remote peer 2e01c23563984d0682838b27189624fa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:10:48.323513 1707 consensus_queue.cc:1035] T c8fabc3af751454c89f470d4c1e8f915 P 2e01c23563984d0682838b27189624fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2c00b8f050b47fabe1abc7847e2db48" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 40101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:10:48.323701 1698 consensus_queue.cc:1035] T c8fabc3af751454c89f470d4c1e8f915 P 2e01c23563984d0682838b27189624fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "599840b07db24ad088880d0fdc1c5cba" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 44967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:10:48.325585 1693 tablet_replica.cc:439] T c8fabc3af751454c89f470d4c1e8f915 P 599840b07db24ad088880d0fdc1c5cba: TxnStatusTablet state changed. Reason: New leader 2e01c23563984d0682838b27189624fa. Latest consensus state: current_term: 1 leader_uuid: "2e01c23563984d0682838b27189624fa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e01c23563984d0682838b27189624fa" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 42565 } } peers { permanent_uuid: "599840b07db24ad088880d0fdc1c5cba" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 44967 } } peers { permanent_uuid: "b2c00b8f050b47fabe1abc7847e2db48" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 40101 } } }
I20260805 23:10:48.325721 1693 tablet_replica.cc:442] T c8fabc3af751454c89f470d4c1e8f915 P 599840b07db24ad088880d0fdc1c5cba: This TxnStatusTablet replica's current role is: FOLLOWER
I20260805 23:10:48.325678 1694 tablet_replica.cc:439] T c8fabc3af751454c89f470d4c1e8f915 P b2c00b8f050b47fabe1abc7847e2db48: TxnStatusTablet state changed. Reason: New leader 2e01c23563984d0682838b27189624fa. Latest consensus state: current_term: 1 leader_uuid: "2e01c23563984d0682838b27189624fa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e01c23563984d0682838b27189624fa" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 42565 } } peers { permanent_uuid: "599840b07db24ad088880d0fdc1c5cba" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 44967 } } peers { permanent_uuid: "b2c00b8f050b47fabe1abc7847e2db48" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 40101 } } }
I20260805 23:10:48.325788 1694 tablet_replica.cc:442] T c8fabc3af751454c89f470d4c1e8f915 P b2c00b8f050b47fabe1abc7847e2db48: This TxnStatusTablet replica's current role is: FOLLOWER
I20260805 23:10:48.327538 1707 tablet_replica.cc:439] T c8fabc3af751454c89f470d4c1e8f915 P 2e01c23563984d0682838b27189624fa: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2e01c23563984d0682838b27189624fa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e01c23563984d0682838b27189624fa" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 42565 } } peers { permanent_uuid: "599840b07db24ad088880d0fdc1c5cba" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 44967 } } peers { permanent_uuid: "b2c00b8f050b47fabe1abc7847e2db48" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 40101 } } }
I20260805 23:10:48.327715 1707 tablet_replica.cc:442] T c8fabc3af751454c89f470d4c1e8f915 P 2e01c23563984d0682838b27189624fa: This TxnStatusTablet replica's current role is: LEADER
I20260805 23:10:48.328819 1698 tablet_replica.cc:439] T c8fabc3af751454c89f470d4c1e8f915 P 2e01c23563984d0682838b27189624fa: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2e01c23563984d0682838b27189624fa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e01c23563984d0682838b27189624fa" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 42565 } } peers { permanent_uuid: "599840b07db24ad088880d0fdc1c5cba" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 44967 } } peers { permanent_uuid: "b2c00b8f050b47fabe1abc7847e2db48" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 40101 } } }
I20260805 23:10:48.328933 1698 tablet_replica.cc:442] T c8fabc3af751454c89f470d4c1e8f915 P 2e01c23563984d0682838b27189624fa: This TxnStatusTablet replica's current role is: LEADER
I20260805 23:10:48.329842 1694 tablet_replica.cc:439] T c8fabc3af751454c89f470d4c1e8f915 P b2c00b8f050b47fabe1abc7847e2db48: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2e01c23563984d0682838b27189624fa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e01c23563984d0682838b27189624fa" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 42565 } } peers { permanent_uuid: "599840b07db24ad088880d0fdc1c5cba" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 44967 } } peers { permanent_uuid: "b2c00b8f050b47fabe1abc7847e2db48" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 40101 } } }
I20260805 23:10:48.329962 1694 tablet_replica.cc:442] T c8fabc3af751454c89f470d4c1e8f915 P b2c00b8f050b47fabe1abc7847e2db48: This TxnStatusTablet replica's current role is: FOLLOWER
I20260805 23:10:48.331156 1693 tablet_replica.cc:439] T c8fabc3af751454c89f470d4c1e8f915 P 599840b07db24ad088880d0fdc1c5cba: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2e01c23563984d0682838b27189624fa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e01c23563984d0682838b27189624fa" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 42565 } } peers { permanent_uuid: "599840b07db24ad088880d0fdc1c5cba" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 44967 } } peers { permanent_uuid: "b2c00b8f050b47fabe1abc7847e2db48" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 40101 } } }
I20260805 23:10:48.331300 1693 tablet_replica.cc:442] T c8fabc3af751454c89f470d4c1e8f915 P 599840b07db24ad088880d0fdc1c5cba: This TxnStatusTablet replica's current role is: FOLLOWER
I20260805 23:10:48.337618 1682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:48.333113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34870 (local address 127.25.92.195:40101)
0805 23:10:48.333225 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:48.333230 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:48.333243 (+ 13us) server_negotiation.cc:407] Connection header received
0805 23:10:48.333333 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:48.333337 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:48.333456 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:48.333600 (+ 144us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 23:10:48.333945 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:48.334743 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:48.335913 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:48.336164 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:48.336279 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:48.336629 (+ 350us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 23:10:48.337190 (+ 561us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 23:10:48.337324 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:48.337390 (+ 66us) server_negotiation.cc:299] Negotiation successful
0805 23:10:48.337452 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260805 23:10:48.350023 1684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:48.345894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.195:51053 (local address 127.25.92.193:42565)
0805 23:10:48.346045 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:48.346049 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:48.346181 (+ 132us) server_negotiation.cc:407] Connection header received
0805 23:10:48.346325 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:48.346327 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:48.346375 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:48.346440 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 23:10:48.346761 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:48.347339 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:48.348599 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:48.348842 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:48.348935 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:48.349286 (+ 351us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 23:10:48.349362 (+ 76us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 23:10:48.349419 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:48.349691 (+ 272us) server_negotiation.cc:299] Negotiation successful
0805 23:10:48.349764 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260805 23:10:48.350226 1739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:48.345747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.193:42565 (local address 127.25.92.195:51053)
0805 23:10:48.346089 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:48.346104 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:48.346222 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:48.346472 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:48.346475 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:48.346483 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0805 23:10:48.346612 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:48.346618 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:48.347458 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:48.347461 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:48.348458 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:48.348469 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:48.348985 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:48.348989 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:48.349084 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:48.349144 (+ 60us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0805 23:10:48.349446 (+ 302us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0805 23:10:48.349452 (+ 6us) client_negotiation.cc:769] Sending connection context
0805 23:10:48.349537 (+ 85us) client_negotiation.cc:240] Negotiation successful
0805 23:10:48.349624 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":115,"threads_started":1}
I20260805 23:10:48.358444 1466 raft_consensus.cc:1261] T 51693ca5017442ba8da4bda95255cd59 P 599840b07db24ad088880d0fdc1c5cba [term 1 FOLLOWER]: Refusing update from remote peer b2c00b8f050b47fabe1abc7847e2db48: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:10:48.358637 1323 raft_consensus.cc:1261] T 51693ca5017442ba8da4bda95255cd59 P 2e01c23563984d0682838b27189624fa [term 1 FOLLOWER]: Refusing update from remote peer b2c00b8f050b47fabe1abc7847e2db48: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:10:48.358994 1719 consensus_queue.cc:1035] T 51693ca5017442ba8da4bda95255cd59 P b2c00b8f050b47fabe1abc7847e2db48 [LEADER]: Connected to new peer: Peer: permanent_uuid: "599840b07db24ad088880d0fdc1c5cba" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 44967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:10:48.359193 1694 consensus_queue.cc:1035] T 51693ca5017442ba8da4bda95255cd59 P b2c00b8f050b47fabe1abc7847e2db48 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2e01c23563984d0682838b27189624fa" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 42565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:10:48.385533 1682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:48.380658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.193:39411 (local address 127.25.92.195:40101)
0805 23:10:48.380977 (+ 319us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:48.380984 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:48.381267 (+ 283us) server_negotiation.cc:407] Connection header received
0805 23:10:48.381467 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:48.381471 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:48.381552 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:48.381601 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 23:10:48.382046 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:48.382810 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:48.384129 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:48.384459 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:48.384566 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:48.384970 (+ 404us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 23:10:48.385061 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 23:10:48.385110 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:48.385283 (+ 173us) server_negotiation.cc:299] Negotiation successful
0805 23:10:48.385345 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":220}
I20260805 23:10:48.386165 1743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:48.380617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.195:40101 (local address 127.25.92.193:39411)
0805 23:10:48.381188 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:48.381204 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:48.381345 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:48.381747 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:48.381749 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:48.381757 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0805 23:10:48.381861 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:48.381870 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:48.383378 (+ 1508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:48.383380 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:48.384001 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:48.384007 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:48.384659 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:48.384663 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:48.384749 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:48.384815 (+ 66us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0805 23:10:48.385176 (+ 361us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0805 23:10:48.385182 (+ 6us) client_negotiation.cc:769] Sending connection context
0805 23:10:48.385926 (+ 744us) client_negotiation.cc:240] Negotiation successful
0805 23:10:48.386005 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":115,"threads_started":1}
I20260805 23:10:48.418637 1162 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44170:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:10:48.418831 1162 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44170:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:10:48.421247 1162 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3d3f991025e246978b016ce9b762cfa0: Sending DeleteTablet for 3 replicas of tablet 51693ca5017442ba8da4bda95255cd59
I20260805 23:10:48.422043 1446 tablet_service.cc:1514] Processing DeleteTablet for tablet 51693ca5017442ba8da4bda95255cd59 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:10:48 UTC) from {username='oryx', principal='oryx/127.25.92.254@KRBTEST.COM'} at 127.0.0.1:60674
I20260805 23:10:48.422217 1303 tablet_service.cc:1514] Processing DeleteTablet for tablet 51693ca5017442ba8da4bda95255cd59 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:10:48 UTC) from {username='oryx', principal='oryx/127.25.92.254@KRBTEST.COM'} at 127.0.0.1:47526
I20260805 23:10:48.422290 1711 tablet_replica.cc:330] T 51693ca5017442ba8da4bda95255cd59 P 599840b07db24ad088880d0fdc1c5cba: stopping tablet replica
I20260805 23:10:48.422408 1711 raft_consensus.cc:2229] T 51693ca5017442ba8da4bda95255cd59 P 599840b07db24ad088880d0fdc1c5cba [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 23:10:48.422420 1590 tablet_service.cc:1514] Processing DeleteTablet for tablet 51693ca5017442ba8da4bda95255cd59 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:10:48 UTC) from {username='oryx', principal='oryx/127.25.92.254@KRBTEST.COM'} at 127.0.0.1:34862
I20260805 23:10:48.422550 1711 raft_consensus.cc:2258] T 51693ca5017442ba8da4bda95255cd59 P 599840b07db24ad088880d0fdc1c5cba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:10:48.422580 1713 tablet_replica.cc:330] T 51693ca5017442ba8da4bda95255cd59 P b2c00b8f050b47fabe1abc7847e2db48: stopping tablet replica
I20260805 23:10:48.422698 1713 raft_consensus.cc:2229] T 51693ca5017442ba8da4bda95255cd59 P b2c00b8f050b47fabe1abc7847e2db48 [term 1 LEADER]: Raft consensus shutting down.
I20260805 23:10:48.422726 1712 tablet_replica.cc:330] T 51693ca5017442ba8da4bda95255cd59 P 2e01c23563984d0682838b27189624fa: stopping tablet replica
I20260805 23:10:48.422829 1712 raft_consensus.cc:2229] T 51693ca5017442ba8da4bda95255cd59 P 2e01c23563984d0682838b27189624fa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 23:10:48.422948 1712 raft_consensus.cc:2258] T 51693ca5017442ba8da4bda95255cd59 P 2e01c23563984d0682838b27189624fa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:10:48.423017 1713 raft_consensus.cc:2258] T 51693ca5017442ba8da4bda95255cd59 P b2c00b8f050b47fabe1abc7847e2db48 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:10:48.423825 1713 ts_tablet_manager.cc:1905] T 51693ca5017442ba8da4bda95255cd59 P b2c00b8f050b47fabe1abc7847e2db48: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:10:48.423888 1712 ts_tablet_manager.cc:1905] T 51693ca5017442ba8da4bda95255cd59 P 2e01c23563984d0682838b27189624fa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:10:48.427094 1713 ts_tablet_manager.cc:1918] T 51693ca5017442ba8da4bda95255cd59 P b2c00b8f050b47fabe1abc7847e2db48: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260805 23:10:48.427183 1713 log.cc:1196] T 51693ca5017442ba8da4bda95255cd59 P b2c00b8f050b47fabe1abc7847e2db48: Deleting WAL directory at /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/minicluster-data/ts-2/wal/wals/51693ca5017442ba8da4bda95255cd59
I20260805 23:10:48.427436 1712 ts_tablet_manager.cc:1918] T 51693ca5017442ba8da4bda95255cd59 P 2e01c23563984d0682838b27189624fa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260805 23:10:48.427495 1713 ts_tablet_manager.cc:1939] T 51693ca5017442ba8da4bda95255cd59 P b2c00b8f050b47fabe1abc7847e2db48: Deleting consensus metadata
I20260805 23:10:48.427516 1712 log.cc:1196] T 51693ca5017442ba8da4bda95255cd59 P 2e01c23563984d0682838b27189624fa: Deleting WAL directory at /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/minicluster-data/ts-0/wal/wals/51693ca5017442ba8da4bda95255cd59
I20260805 23:10:48.427801 1712 ts_tablet_manager.cc:1939] T 51693ca5017442ba8da4bda95255cd59 P 2e01c23563984d0682838b27189624fa: Deleting consensus metadata
I20260805 23:10:48.428316 1144 catalog_manager.cc:4841] TS b2c00b8f050b47fabe1abc7847e2db48 (127.25.92.195:40101): tablet 51693ca5017442ba8da4bda95255cd59 (table test-table [id=4c4b138d75a04d3a9f3baa0d14754977]) successfully deleted
I20260805 23:10:48.428562 1147 catalog_manager.cc:4841] TS 2e01c23563984d0682838b27189624fa (127.25.92.193:42565): tablet 51693ca5017442ba8da4bda95255cd59 (table test-table [id=4c4b138d75a04d3a9f3baa0d14754977]) successfully deleted
I20260805 23:10:48.429284 1711 ts_tablet_manager.cc:1905] T 51693ca5017442ba8da4bda95255cd59 P 599840b07db24ad088880d0fdc1c5cba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:10:48.432881 1711 ts_tablet_manager.cc:1918] T 51693ca5017442ba8da4bda95255cd59 P 599840b07db24ad088880d0fdc1c5cba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260805 23:10:48.432963 1711 log.cc:1196] T 51693ca5017442ba8da4bda95255cd59 P 599840b07db24ad088880d0fdc1c5cba: Deleting WAL directory at /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/minicluster-data/ts-1/wal/wals/51693ca5017442ba8da4bda95255cd59
I20260805 23:10:48.433225 1711 ts_tablet_manager.cc:1939] T 51693ca5017442ba8da4bda95255cd59 P 599840b07db24ad088880d0fdc1c5cba: Deleting consensus metadata
I20260805 23:10:48.434239 1144 catalog_manager.cc:4841] TS 599840b07db24ad088880d0fdc1c5cba (127.25.92.194:44967): tablet 51693ca5017442ba8da4bda95255cd59 (table test-table [id=4c4b138d75a04d3a9f3baa0d14754977]) successfully deleted
Aug 05 23:10:48 dist-test-slave-ss4n krb5kdc[1113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971448, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 05 23:10:48 dist-test-slave-ss4n krb5kdc[1113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:10:48 dist-test-slave-ss4n krb5kdc[1113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
W20260805 23:10:48.458750 1727 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:10:48.450509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44184 (local address 127.25.92.254:38875)
0805 23:10:48.450645 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:48.450651 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:48.450930 (+ 279us) server_negotiation.cc:407] Connection header received
0805 23:10:48.451045 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:48.451049 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:48.451111 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:48.451223 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:48.452129 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:48.452965 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:48.454266 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:48.454625 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:48.454751 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:48.458604 (+ 3853us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44184: BlockingRecv error: recv got EOF from 127.0.0.1:44184 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 05 23:10:48 dist-test-slave-ss4n krb5kdc[1113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971448, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM
I20260805 23:10:48.471179 1727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:48.460421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44192 (local address 127.25.92.254:38875)
0805 23:10:48.460552 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:48.460556 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:48.461031 (+ 475us) server_negotiation.cc:407] Connection header received
0805 23:10:48.461138 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:48.461141 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:48.461206 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:48.461300 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:48.462066 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:48.462886 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:48.464098 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:48.464435 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:48.464541 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:48.467510 (+ 2969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:48.467515 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:48.467518 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:48.467548 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:48.469244 (+ 1696us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:48.469776 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:48.469782 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:48.469785 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:48.469878 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:48.470233 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:48.470237 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:48.470239 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:48.470470 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:48.470599 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:48.470878 (+ 279us) server_negotiation.cc:299] Negotiation successful
0805 23:10:48.471008 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260805 23:10:48.474288 1162 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44192:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260805 23:10:48.474777 1162 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260805 23:10:48.480067 1303 tablet_service.cc:1467] Processing CreateTablet for tablet 6e5ce6ad374b4082a875a510e94a29f1 (DEFAULT_TABLE table=test-table [id=5b3c9311dbfc4a36a6f94dab13914242]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:10:48.480284 1446 tablet_service.cc:1467] Processing CreateTablet for tablet 6e5ce6ad374b4082a875a510e94a29f1 (DEFAULT_TABLE table=test-table [id=5b3c9311dbfc4a36a6f94dab13914242]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:10:48.480374 1303 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e5ce6ad374b4082a875a510e94a29f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:10:48.480566 1446 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e5ce6ad374b4082a875a510e94a29f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:10:48.481942 1590 tablet_service.cc:1467] Processing CreateTablet for tablet 6e5ce6ad374b4082a875a510e94a29f1 (DEFAULT_TABLE table=test-table [id=5b3c9311dbfc4a36a6f94dab13914242]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:10:48.482240 1590 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e5ce6ad374b4082a875a510e94a29f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:10:48.482893 1686 tablet_bootstrap.cc:492] T 6e5ce6ad374b4082a875a510e94a29f1 P 2e01c23563984d0682838b27189624fa: Bootstrap starting.
I20260805 23:10:48.483760 1686 tablet_bootstrap.cc:654] T 6e5ce6ad374b4082a875a510e94a29f1 P 2e01c23563984d0682838b27189624fa: Neither blocks nor log segments found. Creating new log.
I20260805 23:10:48.484607 1688 tablet_bootstrap.cc:492] T 6e5ce6ad374b4082a875a510e94a29f1 P b2c00b8f050b47fabe1abc7847e2db48: Bootstrap starting.
I20260805 23:10:48.485541 1688 tablet_bootstrap.cc:654] T 6e5ce6ad374b4082a875a510e94a29f1 P b2c00b8f050b47fabe1abc7847e2db48: Neither blocks nor log segments found. Creating new log.
I20260805 23:10:48.486624 1687 tablet_bootstrap.cc:492] T 6e5ce6ad374b4082a875a510e94a29f1 P 599840b07db24ad088880d0fdc1c5cba: Bootstrap starting.
I20260805 23:10:48.487411 1687 tablet_bootstrap.cc:654] T 6e5ce6ad374b4082a875a510e94a29f1 P 599840b07db24ad088880d0fdc1c5cba: Neither blocks nor log segments found. Creating new log.
I20260805 23:10:48.487923 1688 tablet_bootstrap.cc:492] T 6e5ce6ad374b4082a875a510e94a29f1 P b2c00b8f050b47fabe1abc7847e2db48: No bootstrap required, opened a new log
I20260805 23:10:48.488005 1688 ts_tablet_manager.cc:1397] T 6e5ce6ad374b4082a875a510e94a29f1 P b2c00b8f050b47fabe1abc7847e2db48: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260805 23:10:48.488453 1688 raft_consensus.cc:348] T 6e5ce6ad374b4082a875a510e94a29f1 P b2c00b8f050b47fabe1abc7847e2db48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e01c23563984d0682838b27189624fa" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 42565 } } peers { permanent_uuid: "599840b07db24ad088880d0fdc1c5cba" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 44967 } } peers { permanent_uuid: "b2c00b8f050b47fabe1abc7847e2db48" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 40101 } }
I20260805 23:10:48.488610 1688 raft_consensus.cc:374] T 6e5ce6ad374b4082a875a510e94a29f1 P b2c00b8f050b47fabe1abc7847e2db48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:10:48.488646 1688 raft_consensus.cc:729] T 6e5ce6ad374b4082a875a510e94a29f1 P b2c00b8f050b47fabe1abc7847e2db48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2c00b8f050b47fabe1abc7847e2db48, State: Initialized, Role: FOLLOWER
I20260805 23:10:48.488766 1688 consensus_queue.cc:260] T 6e5ce6ad374b4082a875a510e94a29f1 P b2c00b8f050b47fabe1abc7847e2db48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e01c23563984d0682838b27189624fa" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 42565 } } peers { permanent_uuid: "599840b07db24ad088880d0fdc1c5cba" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 44967 } } peers { permanent_uuid: "b2c00b8f050b47fabe1abc7847e2db48" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 40101 } }
I20260805 23:10:48.489037 1688 ts_tablet_manager.cc:1428] T 6e5ce6ad374b4082a875a510e94a29f1 P b2c00b8f050b47fabe1abc7847e2db48: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260805 23:10:48.489750 1687 tablet_bootstrap.cc:492] T 6e5ce6ad374b4082a875a510e94a29f1 P 599840b07db24ad088880d0fdc1c5cba: No bootstrap required, opened a new log
I20260805 23:10:48.489830 1687 ts_tablet_manager.cc:1397] T 6e5ce6ad374b4082a875a510e94a29f1 P 599840b07db24ad088880d0fdc1c5cba: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260805 23:10:48.490403 1687 raft_consensus.cc:348] T 6e5ce6ad374b4082a875a510e94a29f1 P 599840b07db24ad088880d0fdc1c5cba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e01c23563984d0682838b27189624fa" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 42565 } } peers { permanent_uuid: "599840b07db24ad088880d0fdc1c5cba" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 44967 } } peers { permanent_uuid: "b2c00b8f050b47fabe1abc7847e2db48" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 40101 } }
I20260805 23:10:48.490517 1687 raft_consensus.cc:374] T 6e5ce6ad374b4082a875a510e94a29f1 P 599840b07db24ad088880d0fdc1c5cba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:10:48.490547 1687 raft_consensus.cc:729] T 6e5ce6ad374b4082a875a510e94a29f1 P 599840b07db24ad088880d0fdc1c5cba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 599840b07db24ad088880d0fdc1c5cba, State: Initialized, Role: FOLLOWER
I20260805 23:10:48.490658 1687 consensus_queue.cc:260] T 6e5ce6ad374b4082a875a510e94a29f1 P 599840b07db24ad088880d0fdc1c5cba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e01c23563984d0682838b27189624fa" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 42565 } } peers { permanent_uuid: "599840b07db24ad088880d0fdc1c5cba" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 44967 } } peers { permanent_uuid: "b2c00b8f050b47fabe1abc7847e2db48" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 40101 } }
I20260805 23:10:48.490880 1687 ts_tablet_manager.cc:1428] T 6e5ce6ad374b4082a875a510e94a29f1 P 599840b07db24ad088880d0fdc1c5cba: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260805 23:10:48.491549 1686 tablet_bootstrap.cc:492] T 6e5ce6ad374b4082a875a510e94a29f1 P 2e01c23563984d0682838b27189624fa: No bootstrap required, opened a new log
I20260805 23:10:48.491643 1686 ts_tablet_manager.cc:1397] T 6e5ce6ad374b4082a875a510e94a29f1 P 2e01c23563984d0682838b27189624fa: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.001s
I20260805 23:10:48.492008 1686 raft_consensus.cc:348] T 6e5ce6ad374b4082a875a510e94a29f1 P 2e01c23563984d0682838b27189624fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e01c23563984d0682838b27189624fa" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 42565 } } peers { permanent_uuid: "599840b07db24ad088880d0fdc1c5cba" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 44967 } } peers { permanent_uuid: "b2c00b8f050b47fabe1abc7847e2db48" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 40101 } }
I20260805 23:10:48.492118 1686 raft_consensus.cc:374] T 6e5ce6ad374b4082a875a510e94a29f1 P 2e01c23563984d0682838b27189624fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:10:48.492149 1686 raft_consensus.cc:729] T 6e5ce6ad374b4082a875a510e94a29f1 P 2e01c23563984d0682838b27189624fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e01c23563984d0682838b27189624fa, State: Initialized, Role: FOLLOWER
I20260805 23:10:48.492259 1686 consensus_queue.cc:260] T 6e5ce6ad374b4082a875a510e94a29f1 P 2e01c23563984d0682838b27189624fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e01c23563984d0682838b27189624fa" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 42565 } } peers { permanent_uuid: "599840b07db24ad088880d0fdc1c5cba" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 44967 } } peers { permanent_uuid: "b2c00b8f050b47fabe1abc7847e2db48" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 40101 } }
I20260805 23:10:48.492493 1686 ts_tablet_manager.cc:1428] T 6e5ce6ad374b4082a875a510e94a29f1 P 2e01c23563984d0682838b27189624fa: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260805 23:10:48.642432 1744 raft_consensus.cc:482] T 6e5ce6ad374b4082a875a510e94a29f1 P b2c00b8f050b47fabe1abc7847e2db48 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 23:10:48.642647 1744 raft_consensus.cc:504] T 6e5ce6ad374b4082a875a510e94a29f1 P b2c00b8f050b47fabe1abc7847e2db48 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e01c23563984d0682838b27189624fa" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 42565 } } peers { permanent_uuid: "599840b07db24ad088880d0fdc1c5cba" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 44967 } } peers { permanent_uuid: "b2c00b8f050b47fabe1abc7847e2db48" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 40101 } }
I20260805 23:10:48.643193 1744 leader_election.cc:288] T 6e5ce6ad374b4082a875a510e94a29f1 P b2c00b8f050b47fabe1abc7847e2db48 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2e01c23563984d0682838b27189624fa (127.25.92.193:42565), 599840b07db24ad088880d0fdc1c5cba (127.25.92.194:44967)
I20260805 23:10:48.643661 1322 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e5ce6ad374b4082a875a510e94a29f1" candidate_uuid: "b2c00b8f050b47fabe1abc7847e2db48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e01c23563984d0682838b27189624fa" is_pre_election: true
I20260805 23:10:48.643637 1465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e5ce6ad374b4082a875a510e94a29f1" candidate_uuid: "b2c00b8f050b47fabe1abc7847e2db48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "599840b07db24ad088880d0fdc1c5cba" is_pre_election: true
I20260805 23:10:48.643850 1465 raft_consensus.cc:2454] T 6e5ce6ad374b4082a875a510e94a29f1 P 599840b07db24ad088880d0fdc1c5cba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2c00b8f050b47fabe1abc7847e2db48 in term 0.
I20260805 23:10:48.643844 1322 raft_consensus.cc:2454] T 6e5ce6ad374b4082a875a510e94a29f1 P 2e01c23563984d0682838b27189624fa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2c00b8f050b47fabe1abc7847e2db48 in term 0.
I20260805 23:10:48.644158 1539 leader_election.cc:302] T 6e5ce6ad374b4082a875a510e94a29f1 P b2c00b8f050b47fabe1abc7847e2db48 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 599840b07db24ad088880d0fdc1c5cba, b2c00b8f050b47fabe1abc7847e2db48; no voters:
I20260805 23:10:48.644367 1744 raft_consensus.cc:2781] T 6e5ce6ad374b4082a875a510e94a29f1 P b2c00b8f050b47fabe1abc7847e2db48 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260805 23:10:48.644433 1744 raft_consensus.cc:482] T 6e5ce6ad374b4082a875a510e94a29f1 P b2c00b8f050b47fabe1abc7847e2db48 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260805 23:10:48.644479 1744 raft_consensus.cc:3037] T 6e5ce6ad374b4082a875a510e94a29f1 P b2c00b8f050b47fabe1abc7847e2db48 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:10:48.646066 1744 raft_consensus.cc:504] T 6e5ce6ad374b4082a875a510e94a29f1 P b2c00b8f050b47fabe1abc7847e2db48 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e01c23563984d0682838b27189624fa" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 42565 } } peers { permanent_uuid: "599840b07db24ad088880d0fdc1c5cba" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 44967 } } peers { permanent_uuid: "b2c00b8f050b47fabe1abc7847e2db48" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 40101 } }
I20260805 23:10:48.646462 1744 leader_election.cc:288] T 6e5ce6ad374b4082a875a510e94a29f1 P b2c00b8f050b47fabe1abc7847e2db48 [CANDIDATE]: Term 1 election: Requested vote from peers 2e01c23563984d0682838b27189624fa (127.25.92.193:42565), 599840b07db24ad088880d0fdc1c5cba (127.25.92.194:44967)
I20260805 23:10:48.646904 1322 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e5ce6ad374b4082a875a510e94a29f1" candidate_uuid: "b2c00b8f050b47fabe1abc7847e2db48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e01c23563984d0682838b27189624fa"
I20260805 23:10:48.646898 1465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e5ce6ad374b4082a875a510e94a29f1" candidate_uuid: "b2c00b8f050b47fabe1abc7847e2db48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "599840b07db24ad088880d0fdc1c5cba"
I20260805 23:10:48.647063 1322 raft_consensus.cc:3037] T 6e5ce6ad374b4082a875a510e94a29f1 P 2e01c23563984d0682838b27189624fa [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:10:48.647059 1465 raft_consensus.cc:3037] T 6e5ce6ad374b4082a875a510e94a29f1 P 599840b07db24ad088880d0fdc1c5cba [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:10:48.648120 1465 raft_consensus.cc:2454] T 6e5ce6ad374b4082a875a510e94a29f1 P 599840b07db24ad088880d0fdc1c5cba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2c00b8f050b47fabe1abc7847e2db48 in term 1.
I20260805 23:10:48.648362 1322 raft_consensus.cc:2454] T 6e5ce6ad374b4082a875a510e94a29f1 P 2e01c23563984d0682838b27189624fa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2c00b8f050b47fabe1abc7847e2db48 in term 1.
I20260805 23:10:48.648548 1539 leader_election.cc:302] T 6e5ce6ad374b4082a875a510e94a29f1 P b2c00b8f050b47fabe1abc7847e2db48 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 599840b07db24ad088880d0fdc1c5cba, b2c00b8f050b47fabe1abc7847e2db48; no voters:
I20260805 23:10:48.648799 1744 raft_consensus.cc:2781] T 6e5ce6ad374b4082a875a510e94a29f1 P b2c00b8f050b47fabe1abc7847e2db48 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:10:48.648916 1744 raft_consensus.cc:686] T 6e5ce6ad374b4082a875a510e94a29f1 P b2c00b8f050b47fabe1abc7847e2db48 [term 1 LEADER]: Becoming Leader. State: Replica: b2c00b8f050b47fabe1abc7847e2db48, State: Running, Role: LEADER
I20260805 23:10:48.649080 1744 consensus_queue.cc:237] T 6e5ce6ad374b4082a875a510e94a29f1 P b2c00b8f050b47fabe1abc7847e2db48 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e01c23563984d0682838b27189624fa" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 42565 } } peers { permanent_uuid: "599840b07db24ad088880d0fdc1c5cba" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 44967 } } peers { permanent_uuid: "b2c00b8f050b47fabe1abc7847e2db48" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 40101 } }
I20260805 23:10:48.650923 1162 catalog_manager.cc:5495] T 6e5ce6ad374b4082a875a510e94a29f1 P b2c00b8f050b47fabe1abc7847e2db48 reported cstate change: term changed from 0 to 1, leader changed from <none> to b2c00b8f050b47fabe1abc7847e2db48 (127.25.92.195). New cstate: current_term: 1 leader_uuid: "b2c00b8f050b47fabe1abc7847e2db48" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e01c23563984d0682838b27189624fa" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 42565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "599840b07db24ad088880d0fdc1c5cba" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 44967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2c00b8f050b47fabe1abc7847e2db48" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 40101 } health_report { overall_health: HEALTHY } } }
I20260805 23:10:48.688632 1682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:48.685200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34876 (local address 127.25.92.195:40101)
0805 23:10:48.685355 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:48.685360 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:48.685380 (+ 20us) server_negotiation.cc:407] Connection header received
0805 23:10:48.685453 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:48.685456 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:48.685564 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:48.685657 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 23:10:48.685957 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:48.686523 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:48.687475 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:48.687717 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:48.687841 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:48.688120 (+ 279us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 23:10:48.688195 (+ 75us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 23:10:48.688282 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:48.688422 (+ 140us) server_negotiation.cc:299] Negotiation successful
0805 23:10:48.688474 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260805 23:10:48.691009 1465 raft_consensus.cc:1261] T 6e5ce6ad374b4082a875a510e94a29f1 P 599840b07db24ad088880d0fdc1c5cba [term 1 FOLLOWER]: Refusing update from remote peer b2c00b8f050b47fabe1abc7847e2db48: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:10:48.691071 1322 raft_consensus.cc:1261] T 6e5ce6ad374b4082a875a510e94a29f1 P 2e01c23563984d0682838b27189624fa [term 1 FOLLOWER]: Refusing update from remote peer b2c00b8f050b47fabe1abc7847e2db48: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:10:48.691486 1744 consensus_queue.cc:1035] T 6e5ce6ad374b4082a875a510e94a29f1 P b2c00b8f050b47fabe1abc7847e2db48 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2e01c23563984d0682838b27189624fa" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 42565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:10:48.691642 1719 consensus_queue.cc:1035] T 6e5ce6ad374b4082a875a510e94a29f1 P b2c00b8f050b47fabe1abc7847e2db48 [LEADER]: Connected to new peer: Peer: permanent_uuid: "599840b07db24ad088880d0fdc1c5cba" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 44967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:10:48.699285 1158 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44192:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:10:48.699433 1158 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44192:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:10:48.701381 1158 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3d3f991025e246978b016ce9b762cfa0: Sending DeleteTablet for 3 replicas of tablet 6e5ce6ad374b4082a875a510e94a29f1
I20260805 23:10:48.702155 1590 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e5ce6ad374b4082a875a510e94a29f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:10:48 UTC) from {username='oryx', principal='oryx/127.25.92.254@KRBTEST.COM'} at 127.0.0.1:34862
I20260805 23:10:48.702251 1303 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e5ce6ad374b4082a875a510e94a29f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:10:48 UTC) from {username='oryx', principal='oryx/127.25.92.254@KRBTEST.COM'} at 127.0.0.1:47526
I20260805 23:10:48.702414 1713 tablet_replica.cc:330] T 6e5ce6ad374b4082a875a510e94a29f1 P b2c00b8f050b47fabe1abc7847e2db48: stopping tablet replica
I20260805 23:10:48.702502 1446 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e5ce6ad374b4082a875a510e94a29f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:10:48 UTC) from {username='oryx', principal='oryx/127.25.92.254@KRBTEST.COM'} at 127.0.0.1:60674
I20260805 23:10:48.702555 1713 raft_consensus.cc:2229] T 6e5ce6ad374b4082a875a510e94a29f1 P b2c00b8f050b47fabe1abc7847e2db48 [term 1 LEADER]: Raft consensus shutting down.
I20260805 23:10:48.702703 1711 tablet_replica.cc:330] T 6e5ce6ad374b4082a875a510e94a29f1 P 599840b07db24ad088880d0fdc1c5cba: stopping tablet replica
I20260805 23:10:48.702723 1712 tablet_replica.cc:330] T 6e5ce6ad374b4082a875a510e94a29f1 P 2e01c23563984d0682838b27189624fa: stopping tablet replica
I20260805 23:10:48.702787 1713 raft_consensus.cc:2258] T 6e5ce6ad374b4082a875a510e94a29f1 P b2c00b8f050b47fabe1abc7847e2db48 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:10:48.702812 1711 raft_consensus.cc:2229] T 6e5ce6ad374b4082a875a510e94a29f1 P 599840b07db24ad088880d0fdc1c5cba [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 23:10:48.702824 1712 raft_consensus.cc:2229] T 6e5ce6ad374b4082a875a510e94a29f1 P 2e01c23563984d0682838b27189624fa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 23:10:48.702934 1712 raft_consensus.cc:2258] T 6e5ce6ad374b4082a875a510e94a29f1 P 2e01c23563984d0682838b27189624fa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:10:48.702934 1711 raft_consensus.cc:2258] T 6e5ce6ad374b4082a875a510e94a29f1 P 599840b07db24ad088880d0fdc1c5cba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:10:48.702924 1158 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44192:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260805 23:10:48.703646 1713 ts_tablet_manager.cc:1905] T 6e5ce6ad374b4082a875a510e94a29f1 P b2c00b8f050b47fabe1abc7847e2db48: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:10:48.703724 1711 ts_tablet_manager.cc:1905] T 6e5ce6ad374b4082a875a510e94a29f1 P 599840b07db24ad088880d0fdc1c5cba: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260805 23:10:48.703732 1158 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260805 23:10:48.703963 1712 ts_tablet_manager.cc:1905] T 6e5ce6ad374b4082a875a510e94a29f1 P 2e01c23563984d0682838b27189624fa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:10:48.706341 1712 ts_tablet_manager.cc:1918] T 6e5ce6ad374b4082a875a510e94a29f1 P 2e01c23563984d0682838b27189624fa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 23:10:48.706421 1712 log.cc:1196] T 6e5ce6ad374b4082a875a510e94a29f1 P 2e01c23563984d0682838b27189624fa: Deleting WAL directory at /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/minicluster-data/ts-0/wal/wals/6e5ce6ad374b4082a875a510e94a29f1
I20260805 23:10:48.706406 1711 ts_tablet_manager.cc:1918] T 6e5ce6ad374b4082a875a510e94a29f1 P 599840b07db24ad088880d0fdc1c5cba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 23:10:48.706480 1711 log.cc:1196] T 6e5ce6ad374b4082a875a510e94a29f1 P 599840b07db24ad088880d0fdc1c5cba: Deleting WAL directory at /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/minicluster-data/ts-1/wal/wals/6e5ce6ad374b4082a875a510e94a29f1
I20260805 23:10:48.706491 1713 ts_tablet_manager.cc:1918] T 6e5ce6ad374b4082a875a510e94a29f1 P b2c00b8f050b47fabe1abc7847e2db48: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 23:10:48.706588 1713 log.cc:1196] T 6e5ce6ad374b4082a875a510e94a29f1 P b2c00b8f050b47fabe1abc7847e2db48: Deleting WAL directory at /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/minicluster-data/ts-2/wal/wals/6e5ce6ad374b4082a875a510e94a29f1
I20260805 23:10:48.706746 1712 ts_tablet_manager.cc:1939] T 6e5ce6ad374b4082a875a510e94a29f1 P 2e01c23563984d0682838b27189624fa: Deleting consensus metadata
I20260805 23:10:48.706797 1711 ts_tablet_manager.cc:1939] T 6e5ce6ad374b4082a875a510e94a29f1 P 599840b07db24ad088880d0fdc1c5cba: Deleting consensus metadata
I20260805 23:10:48.706883 1713 ts_tablet_manager.cc:1939] T 6e5ce6ad374b4082a875a510e94a29f1 P b2c00b8f050b47fabe1abc7847e2db48: Deleting consensus metadata
I20260805 23:10:48.707595 1147 catalog_manager.cc:4841] TS 2e01c23563984d0682838b27189624fa (127.25.92.193:42565): tablet 6e5ce6ad374b4082a875a510e94a29f1 (table test-table [id=5b3c9311dbfc4a36a6f94dab13914242]) successfully deleted
I20260805 23:10:48.707924 1144 catalog_manager.cc:4841] TS b2c00b8f050b47fabe1abc7847e2db48 (127.25.92.195:40101): tablet 6e5ce6ad374b4082a875a510e94a29f1 (table test-table [id=5b3c9311dbfc4a36a6f94dab13914242]) successfully deleted
I20260805 23:10:48.708082 1144 catalog_manager.cc:4841] TS 599840b07db24ad088880d0fdc1c5cba (127.25.92.194:44967): tablet 6e5ce6ad374b4082a875a510e94a29f1 (table test-table [id=5b3c9311dbfc4a36a6f94dab13914242]) successfully deleted
I20260805 23:10:48.709190 1303 tablet_service.cc:1467] Processing CreateTablet for tablet fa1e5b88bec94ac3a74250b3a6faea42 (DEFAULT_TABLE table=test-table [id=bf335daf38bf4f1bbb8ddc95920e54a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:10:48.709466 1303 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa1e5b88bec94ac3a74250b3a6faea42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:10:48.709551 1590 tablet_service.cc:1467] Processing CreateTablet for tablet fa1e5b88bec94ac3a74250b3a6faea42 (DEFAULT_TABLE table=test-table [id=bf335daf38bf4f1bbb8ddc95920e54a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:10:48.709836 1590 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa1e5b88bec94ac3a74250b3a6faea42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:10:48.711460 1686 tablet_bootstrap.cc:492] T fa1e5b88bec94ac3a74250b3a6faea42 P 2e01c23563984d0682838b27189624fa: Bootstrap starting.
I20260805 23:10:48.711833 1688 tablet_bootstrap.cc:492] T fa1e5b88bec94ac3a74250b3a6faea42 P b2c00b8f050b47fabe1abc7847e2db48: Bootstrap starting.
I20260805 23:10:48.712349 1686 tablet_bootstrap.cc:654] T fa1e5b88bec94ac3a74250b3a6faea42 P 2e01c23563984d0682838b27189624fa: Neither blocks nor log segments found. Creating new log.
I20260805 23:10:48.712658 1688 tablet_bootstrap.cc:654] T fa1e5b88bec94ac3a74250b3a6faea42 P b2c00b8f050b47fabe1abc7847e2db48: Neither blocks nor log segments found. Creating new log.
I20260805 23:10:48.713052 1446 tablet_service.cc:1467] Processing CreateTablet for tablet fa1e5b88bec94ac3a74250b3a6faea42 (DEFAULT_TABLE table=test-table [id=bf335daf38bf4f1bbb8ddc95920e54a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:10:48.713297 1446 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa1e5b88bec94ac3a74250b3a6faea42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:10:48.713598 1686 tablet_bootstrap.cc:492] T fa1e5b88bec94ac3a74250b3a6faea42 P 2e01c23563984d0682838b27189624fa: No bootstrap required, opened a new log
I20260805 23:10:48.713691 1686 ts_tablet_manager.cc:1397] T fa1e5b88bec94ac3a74250b3a6faea42 P 2e01c23563984d0682838b27189624fa: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260805 23:10:48.714152 1686 raft_consensus.cc:348] T fa1e5b88bec94ac3a74250b3a6faea42 P 2e01c23563984d0682838b27189624fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e01c23563984d0682838b27189624fa" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 42565 } } peers { permanent_uuid: "599840b07db24ad088880d0fdc1c5cba" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 44967 } } peers { permanent_uuid: "b2c00b8f050b47fabe1abc7847e2db48" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 40101 } }
I20260805 23:10:48.714291 1686 raft_consensus.cc:374] T fa1e5b88bec94ac3a74250b3a6faea42 P 2e01c23563984d0682838b27189624fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:10:48.714334 1686 raft_consensus.cc:729] T fa1e5b88bec94ac3a74250b3a6faea42 P 2e01c23563984d0682838b27189624fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e01c23563984d0682838b27189624fa, State: Initialized, Role: FOLLOWER
I20260805 23:10:48.714468 1686 consensus_queue.cc:260] T fa1e5b88bec94ac3a74250b3a6faea42 P 2e01c23563984d0682838b27189624fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e01c23563984d0682838b27189624fa" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 42565 } } peers { permanent_uuid: "599840b07db24ad088880d0fdc1c5cba" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 44967 } } peers { permanent_uuid: "b2c00b8f050b47fabe1abc7847e2db48" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 40101 } }
I20260805 23:10:48.714730 1688 tablet_bootstrap.cc:492] T fa1e5b88bec94ac3a74250b3a6faea42 P b2c00b8f050b47fabe1abc7847e2db48: No bootstrap required, opened a new log
I20260805 23:10:48.714766 1686 ts_tablet_manager.cc:1428] T fa1e5b88bec94ac3a74250b3a6faea42 P 2e01c23563984d0682838b27189624fa: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260805 23:10:48.714815 1688 ts_tablet_manager.cc:1397] T fa1e5b88bec94ac3a74250b3a6faea42 P b2c00b8f050b47fabe1abc7847e2db48: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260805 23:10:48.715211 1688 raft_consensus.cc:348] T fa1e5b88bec94ac3a74250b3a6faea42 P b2c00b8f050b47fabe1abc7847e2db48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e01c23563984d0682838b27189624fa" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 42565 } } peers { permanent_uuid: "599840b07db24ad088880d0fdc1c5cba" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 44967 } } peers { permanent_uuid: "b2c00b8f050b47fabe1abc7847e2db48" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 40101 } }
I20260805 23:10:48.715337 1688 raft_consensus.cc:374] T fa1e5b88bec94ac3a74250b3a6faea42 P b2c00b8f050b47fabe1abc7847e2db48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:10:48.715373 1688 raft_consensus.cc:729] T fa1e5b88bec94ac3a74250b3a6faea42 P b2c00b8f050b47fabe1abc7847e2db48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2c00b8f050b47fabe1abc7847e2db48, State: Initialized, Role: FOLLOWER
I20260805 23:10:48.715507 1688 consensus_queue.cc:260] T fa1e5b88bec94ac3a74250b3a6faea42 P b2c00b8f050b47fabe1abc7847e2db48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e01c23563984d0682838b27189624fa" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 42565 } } peers { permanent_uuid: "599840b07db24ad088880d0fdc1c5cba" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 44967 } } peers { permanent_uuid: "b2c00b8f050b47fabe1abc7847e2db48" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 40101 } }
I20260805 23:10:48.715804 1688 ts_tablet_manager.cc:1428] T fa1e5b88bec94ac3a74250b3a6faea42 P b2c00b8f050b47fabe1abc7847e2db48: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260805 23:10:48.715926 1687 tablet_bootstrap.cc:492] T fa1e5b88bec94ac3a74250b3a6faea42 P 599840b07db24ad088880d0fdc1c5cba: Bootstrap starting.
I20260805 23:10:48.716907 1687 tablet_bootstrap.cc:654] T fa1e5b88bec94ac3a74250b3a6faea42 P 599840b07db24ad088880d0fdc1c5cba: Neither blocks nor log segments found. Creating new log.
I20260805 23:10:48.718309 1687 tablet_bootstrap.cc:492] T fa1e5b88bec94ac3a74250b3a6faea42 P 599840b07db24ad088880d0fdc1c5cba: No bootstrap required, opened a new log
I20260805 23:10:48.718384 1687 ts_tablet_manager.cc:1397] T fa1e5b88bec94ac3a74250b3a6faea42 P 599840b07db24ad088880d0fdc1c5cba: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260805 23:10:48.718806 1687 raft_consensus.cc:348] T fa1e5b88bec94ac3a74250b3a6faea42 P 599840b07db24ad088880d0fdc1c5cba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e01c23563984d0682838b27189624fa" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 42565 } } peers { permanent_uuid: "599840b07db24ad088880d0fdc1c5cba" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 44967 } } peers { permanent_uuid: "b2c00b8f050b47fabe1abc7847e2db48" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 40101 } }
I20260805 23:10:48.718926 1687 raft_consensus.cc:374] T fa1e5b88bec94ac3a74250b3a6faea42 P 599840b07db24ad088880d0fdc1c5cba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:10:48.718966 1687 raft_consensus.cc:729] T fa1e5b88bec94ac3a74250b3a6faea42 P 599840b07db24ad088880d0fdc1c5cba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 599840b07db24ad088880d0fdc1c5cba, State: Initialized, Role: FOLLOWER
I20260805 23:10:48.719117 1687 consensus_queue.cc:260] T fa1e5b88bec94ac3a74250b3a6faea42 P 599840b07db24ad088880d0fdc1c5cba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e01c23563984d0682838b27189624fa" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 42565 } } peers { permanent_uuid: "599840b07db24ad088880d0fdc1c5cba" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 44967 } } peers { permanent_uuid: "b2c00b8f050b47fabe1abc7847e2db48" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 40101 } }
I20260805 23:10:48.719429 1687 ts_tablet_manager.cc:1428] T fa1e5b88bec94ac3a74250b3a6faea42 P 599840b07db24ad088880d0fdc1c5cba: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260805 23:10:48.736850 1698 raft_consensus.cc:482] T fa1e5b88bec94ac3a74250b3a6faea42 P 2e01c23563984d0682838b27189624fa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 23:10:48.737011 1698 raft_consensus.cc:504] T fa1e5b88bec94ac3a74250b3a6faea42 P 2e01c23563984d0682838b27189624fa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e01c23563984d0682838b27189624fa" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 42565 } } peers { permanent_uuid: "599840b07db24ad088880d0fdc1c5cba" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 44967 } } peers { permanent_uuid: "b2c00b8f050b47fabe1abc7847e2db48" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 40101 } }
I20260805 23:10:48.737635 1698 leader_election.cc:288] T fa1e5b88bec94ac3a74250b3a6faea42 P 2e01c23563984d0682838b27189624fa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 599840b07db24ad088880d0fdc1c5cba (127.25.92.194:44967), b2c00b8f050b47fabe1abc7847e2db48 (127.25.92.195:40101)
I20260805 23:10:48.738085 1465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa1e5b88bec94ac3a74250b3a6faea42" candidate_uuid: "2e01c23563984d0682838b27189624fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "599840b07db24ad088880d0fdc1c5cba" is_pre_election: true
I20260805 23:10:48.738114 1610 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa1e5b88bec94ac3a74250b3a6faea42" candidate_uuid: "2e01c23563984d0682838b27189624fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2c00b8f050b47fabe1abc7847e2db48" is_pre_election: true
I20260805 23:10:48.738289 1465 raft_consensus.cc:2454] T fa1e5b88bec94ac3a74250b3a6faea42 P 599840b07db24ad088880d0fdc1c5cba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2e01c23563984d0682838b27189624fa in term 0.
I20260805 23:10:48.738296 1610 raft_consensus.cc:2454] T fa1e5b88bec94ac3a74250b3a6faea42 P b2c00b8f050b47fabe1abc7847e2db48 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2e01c23563984d0682838b27189624fa in term 0.
I20260805 23:10:48.738705 1251 leader_election.cc:302] T fa1e5b88bec94ac3a74250b3a6faea42 P 2e01c23563984d0682838b27189624fa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e01c23563984d0682838b27189624fa, 599840b07db24ad088880d0fdc1c5cba; no voters:
I20260805 23:10:48.738947 1698 raft_consensus.cc:2781] T fa1e5b88bec94ac3a74250b3a6faea42 P 2e01c23563984d0682838b27189624fa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260805 23:10:48.739024 1698 raft_consensus.cc:482] T fa1e5b88bec94ac3a74250b3a6faea42 P 2e01c23563984d0682838b27189624fa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260805 23:10:48.739081 1698 raft_consensus.cc:3037] T fa1e5b88bec94ac3a74250b3a6faea42 P 2e01c23563984d0682838b27189624fa [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:10:48.740226 1698 raft_consensus.cc:504] T fa1e5b88bec94ac3a74250b3a6faea42 P 2e01c23563984d0682838b27189624fa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e01c23563984d0682838b27189624fa" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 42565 } } peers { permanent_uuid: "599840b07db24ad088880d0fdc1c5cba" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 44967 } } peers { permanent_uuid: "b2c00b8f050b47fabe1abc7847e2db48" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 40101 } }
I20260805 23:10:48.740564 1698 leader_election.cc:288] T fa1e5b88bec94ac3a74250b3a6faea42 P 2e01c23563984d0682838b27189624fa [CANDIDATE]: Term 1 election: Requested vote from peers 599840b07db24ad088880d0fdc1c5cba (127.25.92.194:44967), b2c00b8f050b47fabe1abc7847e2db48 (127.25.92.195:40101)
I20260805 23:10:48.741020 1465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa1e5b88bec94ac3a74250b3a6faea42" candidate_uuid: "2e01c23563984d0682838b27189624fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "599840b07db24ad088880d0fdc1c5cba"
I20260805 23:10:48.741163 1465 raft_consensus.cc:3037] T fa1e5b88bec94ac3a74250b3a6faea42 P 599840b07db24ad088880d0fdc1c5cba [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:10:48.741119 1610 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa1e5b88bec94ac3a74250b3a6faea42" candidate_uuid: "2e01c23563984d0682838b27189624fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2c00b8f050b47fabe1abc7847e2db48"
I20260805 23:10:48.741443 1610 raft_consensus.cc:3037] T fa1e5b88bec94ac3a74250b3a6faea42 P b2c00b8f050b47fabe1abc7847e2db48 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:10:48.742429 1465 raft_consensus.cc:2454] T fa1e5b88bec94ac3a74250b3a6faea42 P 599840b07db24ad088880d0fdc1c5cba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e01c23563984d0682838b27189624fa in term 1.
I20260805 23:10:48.742672 1610 raft_consensus.cc:2454] T fa1e5b88bec94ac3a74250b3a6faea42 P b2c00b8f050b47fabe1abc7847e2db48 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e01c23563984d0682838b27189624fa in term 1.
I20260805 23:10:48.742806 1251 leader_election.cc:302] T fa1e5b88bec94ac3a74250b3a6faea42 P 2e01c23563984d0682838b27189624fa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e01c23563984d0682838b27189624fa, 599840b07db24ad088880d0fdc1c5cba; no voters:
I20260805 23:10:48.743175 1698 raft_consensus.cc:2781] T fa1e5b88bec94ac3a74250b3a6faea42 P 2e01c23563984d0682838b27189624fa [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:10:48.743281 1698 raft_consensus.cc:686] T fa1e5b88bec94ac3a74250b3a6faea42 P 2e01c23563984d0682838b27189624fa [term 1 LEADER]: Becoming Leader. State: Replica: 2e01c23563984d0682838b27189624fa, State: Running, Role: LEADER
I20260805 23:10:48.743403 1698 consensus_queue.cc:237] T fa1e5b88bec94ac3a74250b3a6faea42 P 2e01c23563984d0682838b27189624fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e01c23563984d0682838b27189624fa" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 42565 } } peers { permanent_uuid: "599840b07db24ad088880d0fdc1c5cba" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 44967 } } peers { permanent_uuid: "b2c00b8f050b47fabe1abc7847e2db48" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 40101 } }
I20260805 23:10:48.745373 1162 catalog_manager.cc:5495] T fa1e5b88bec94ac3a74250b3a6faea42 P 2e01c23563984d0682838b27189624fa reported cstate change: term changed from 0 to 1, leader changed from <none> to 2e01c23563984d0682838b27189624fa (127.25.92.193). New cstate: current_term: 1 leader_uuid: "2e01c23563984d0682838b27189624fa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e01c23563984d0682838b27189624fa" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 42565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "599840b07db24ad088880d0fdc1c5cba" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 44967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2c00b8f050b47fabe1abc7847e2db48" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 40101 } health_report { overall_health: UNKNOWN } } }
I20260805 23:10:48.766713 1684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:48.763950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47540 (local address 127.25.92.193:42565)
0805 23:10:48.764073 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:48.764077 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:48.764089 (+ 12us) server_negotiation.cc:407] Connection header received
0805 23:10:48.764199 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:48.764201 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:48.764242 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:48.764306 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 23:10:48.764519 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:48.765016 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:48.765849 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:48.766060 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:48.766137 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:48.766360 (+ 223us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 23:10:48.766423 (+ 63us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 23:10:48.766474 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:48.766552 (+ 78us) server_negotiation.cc:299] Negotiation successful
0805 23:10:48.766594 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260805 23:10:48.774847 1743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:48.771878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.193:42565 (local address 127.25.92.193:33449)
0805 23:10:48.772039 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:48.772052 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:48.772126 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:48.772361 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:48.772363 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:48.772369 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0805 23:10:48.772447 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:48.772451 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:48.773150 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:48.773153 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:48.773871 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:48.773881 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:48.774287 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:48.774290 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:48.774348 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:48.774373 (+ 25us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0805 23:10:48.774609 (+ 236us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0805 23:10:48.774613 (+ 4us) client_negotiation.cc:769] Sending connection context
0805 23:10:48.774662 (+ 49us) client_negotiation.cc:240] Negotiation successful
0805 23:10:48.774735 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260805 23:10:48.774847 1684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:48.771953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.193:33449 (local address 127.25.92.193:42565)
0805 23:10:48.772055 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:48.772058 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:48.772089 (+ 31us) server_negotiation.cc:407] Connection header received
0805 23:10:48.772208 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:48.772210 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:48.772267 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:48.772325 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 23:10:48.772534 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:48.773060 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:48.773973 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:48.774189 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:48.774269 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:48.774478 (+ 209us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 23:10:48.774538 (+ 60us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 23:10:48.774588 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:48.774669 (+ 81us) server_negotiation.cc:299] Negotiation successful
0805 23:10:48.774713 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":30}
I20260805 23:10:48.780583 1465 raft_consensus.cc:1261] T fa1e5b88bec94ac3a74250b3a6faea42 P 599840b07db24ad088880d0fdc1c5cba [term 1 FOLLOWER]: Refusing update from remote peer 2e01c23563984d0682838b27189624fa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:10:48.780583 1610 raft_consensus.cc:1261] T fa1e5b88bec94ac3a74250b3a6faea42 P b2c00b8f050b47fabe1abc7847e2db48 [term 1 FOLLOWER]: Refusing update from remote peer 2e01c23563984d0682838b27189624fa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:10:48.781064 1692 consensus_queue.cc:1035] T fa1e5b88bec94ac3a74250b3a6faea42 P 2e01c23563984d0682838b27189624fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2c00b8f050b47fabe1abc7847e2db48" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 40101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:10:48.781260 1698 consensus_queue.cc:1035] T fa1e5b88bec94ac3a74250b3a6faea42 P 2e01c23563984d0682838b27189624fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "599840b07db24ad088880d0fdc1c5cba" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 44967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:10:48.815990 1158 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44192:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:10:48.816203 1158 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44192:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:10:48.818584 1158 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3d3f991025e246978b016ce9b762cfa0: Sending DeleteTablet for 3 replicas of tablet fa1e5b88bec94ac3a74250b3a6faea42
I20260805 23:10:48.819180 1303 tablet_service.cc:1514] Processing DeleteTablet for tablet fa1e5b88bec94ac3a74250b3a6faea42 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:10:48 UTC) from {username='oryx', principal='oryx/127.25.92.254@KRBTEST.COM'} at 127.0.0.1:47526
I20260805 23:10:48.819308 1590 tablet_service.cc:1514] Processing DeleteTablet for tablet fa1e5b88bec94ac3a74250b3a6faea42 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:10:48 UTC) from {username='oryx', principal='oryx/127.25.92.254@KRBTEST.COM'} at 127.0.0.1:34862
I20260805 23:10:48.819456 1446 tablet_service.cc:1514] Processing DeleteTablet for tablet fa1e5b88bec94ac3a74250b3a6faea42 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:10:48 UTC) from {username='oryx', principal='oryx/127.25.92.254@KRBTEST.COM'} at 127.0.0.1:60674
I20260805 23:10:48.819522 1713 tablet_replica.cc:330] T fa1e5b88bec94ac3a74250b3a6faea42 P b2c00b8f050b47fabe1abc7847e2db48: stopping tablet replica
I20260805 23:10:48.819656 1711 tablet_replica.cc:330] T fa1e5b88bec94ac3a74250b3a6faea42 P 599840b07db24ad088880d0fdc1c5cba: stopping tablet replica
I20260805 23:10:48.819656 1713 raft_consensus.cc:2229] T fa1e5b88bec94ac3a74250b3a6faea42 P b2c00b8f050b47fabe1abc7847e2db48 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 23:10:48.819758 1711 raft_consensus.cc:2229] T fa1e5b88bec94ac3a74250b3a6faea42 P 599840b07db24ad088880d0fdc1c5cba [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 23:10:48.819788 1713 raft_consensus.cc:2258] T fa1e5b88bec94ac3a74250b3a6faea42 P b2c00b8f050b47fabe1abc7847e2db48 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:10:48.819865 1711 raft_consensus.cc:2258] T fa1e5b88bec94ac3a74250b3a6faea42 P 599840b07db24ad088880d0fdc1c5cba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:10:48.820274 1712 tablet_replica.cc:330] T fa1e5b88bec94ac3a74250b3a6faea42 P 2e01c23563984d0682838b27189624fa: stopping tablet replica
I20260805 23:10:48.820406 1712 raft_consensus.cc:2229] T fa1e5b88bec94ac3a74250b3a6faea42 P 2e01c23563984d0682838b27189624fa [term 1 LEADER]: Raft consensus shutting down.
I20260805 23:10:48.820559 1713 ts_tablet_manager.cc:1905] T fa1e5b88bec94ac3a74250b3a6faea42 P b2c00b8f050b47fabe1abc7847e2db48: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:10:48.820555 1711 ts_tablet_manager.cc:1905] T fa1e5b88bec94ac3a74250b3a6faea42 P 599840b07db24ad088880d0fdc1c5cba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:10:48.820612 1712 raft_consensus.cc:2258] T fa1e5b88bec94ac3a74250b3a6faea42 P 2e01c23563984d0682838b27189624fa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:10:48.821795 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 1235
I20260805 23:10:48.823422 1711 ts_tablet_manager.cc:1918] T fa1e5b88bec94ac3a74250b3a6faea42 P 599840b07db24ad088880d0fdc1c5cba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260805 23:10:48.823559 1713 ts_tablet_manager.cc:1918] T fa1e5b88bec94ac3a74250b3a6faea42 P b2c00b8f050b47fabe1abc7847e2db48: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260805 23:10:48.823643 1713 log.cc:1196] T fa1e5b88bec94ac3a74250b3a6faea42 P b2c00b8f050b47fabe1abc7847e2db48: Deleting WAL directory at /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/minicluster-data/ts-2/wal/wals/fa1e5b88bec94ac3a74250b3a6faea42
I20260805 23:10:48.823596 1711 log.cc:1196] T fa1e5b88bec94ac3a74250b3a6faea42 P 599840b07db24ad088880d0fdc1c5cba: Deleting WAL directory at /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971406251336-25971-0/minicluster-data/ts-1/wal/wals/fa1e5b88bec94ac3a74250b3a6faea42
I20260805 23:10:48.823935 1713 ts_tablet_manager.cc:1939] T fa1e5b88bec94ac3a74250b3a6faea42 P b2c00b8f050b47fabe1abc7847e2db48: Deleting consensus metadata
I20260805 23:10:48.824067 1711 ts_tablet_manager.cc:1939] T fa1e5b88bec94ac3a74250b3a6faea42 P 599840b07db24ad088880d0fdc1c5cba: Deleting consensus metadata
I20260805 23:10:48.824707 1144 catalog_manager.cc:4841] TS 599840b07db24ad088880d0fdc1c5cba (127.25.92.194:44967): tablet fa1e5b88bec94ac3a74250b3a6faea42 (table test-table [id=bf335daf38bf4f1bbb8ddc95920e54a9]) successfully deleted
I20260805 23:10:48.824906 1144 catalog_manager.cc:4841] TS b2c00b8f050b47fabe1abc7847e2db48 (127.25.92.195:40101): tablet fa1e5b88bec94ac3a74250b3a6faea42 (table test-table [id=bf335daf38bf4f1bbb8ddc95920e54a9]) successfully deleted
W20260805 23:10:48.832175 1147 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.25.92.193:42565 (error 108)
I20260805 23:10:48.832365 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 1379
I20260805 23:10:48.834501 1775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:48.833698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.193:42565 (local address 127.0.0.1:47554)
0805 23:10:48.834267 (+ 569us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:48.834358 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.193:42565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":67,"threads_started":1}
W20260805 23:10:48.835002 1147 catalog_manager.cc:4584] TS 2e01c23563984d0682838b27189624fa (127.25.92.193:42565): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet fa1e5b88bec94ac3a74250b3a6faea42: Network error: Client connection negotiation failed: client connection to 127.25.92.193:42565: connect: Connection refused (error 111)
I20260805 23:10:48.839852 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 1523
I20260805 23:10:48.848637 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 1126
2026-08-05T23:10:48Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3516 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785971406251336-25971-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 23:10:48 dist-test-slave-ss4n krb5kdc[1778](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 23:10:48 dist-test-slave-ss4n krb5kdc[1778](info): set up 2 sockets
Aug 05 23:10:48 dist-test-slave-ss4n krb5kdc[1778](info): commencing operation
krb5kdc: starting...
W20260805 23:10:49.996592 25971 mini_kdc.cc:121] Time spent starting KDC: real 1.121s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785971406251336-25971-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785971406251336-25971-0/krb5kdc/test-user.keytab.
Aug 05 23:10:50 dist-test-slave-ss4n krb5kdc[1778](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971450, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T23:10:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:10:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.92.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.92.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.92.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab.
Entry for principal kudu/127.25.92.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab.
I20260805 23:10:50.134063 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785971406251336-25971-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785971406251336-25971-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785971406251336-25971-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.254:42125
--webserver_interface=127.25.92.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab
--principal=kudu/127.25.92.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:41347
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.92.254:42125
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:10:50.250658 1792 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:10:50.250958 1792 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:10:50.251001 1792 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:10:50.255213 1792 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:10:50.255295 1792 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:10:50.255314 1792 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:10:50.255333 1792 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:10:50.255348 1792 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:10:50.261050 1792 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:41347
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785971406251336-25971-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785971406251336-25971-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.92.254:42125
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.254:42125
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.92.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1792
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785971406251336-25971-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:10:50.262333 1792 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:10:50.263423 1792 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:10:50.272933 1801 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:50.272976 1797 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:50.272975 1798 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:10:50.273088 1792 server_base.cc:1034] running on GCE node
I20260805 23:10:50.273895 1792 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:10:50.275048 1792 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:10:50.276232 1792 hybrid_clock.cc:648] HybridClock initialized: now 1785971450276213 us; error 33 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-05T23:10:50Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1440 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 23:10:50 dist-test-slave-ss4n krb5kdc[1806](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 23:10:50 dist-test-slave-ss4n krb5kdc[1806](info): set up 2 sockets
Aug 05 23:10:50 dist-test-slave-ss4n krb5kdc[1806](info): commencing operation
krb5kdc: starting...
W20260805 23:10:51.383340 25971 mini_kdc.cc:121] Time spent starting KDC: real 1.067s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/krb5kdc/test-user.keytab.
Aug 05 23:10:51 dist-test-slave-ss4n krb5kdc[1806](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971451, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T23:10:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:10:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.92.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.92.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.92.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab.
Entry for principal kudu/127.25.92.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab.
I20260805 23:10:51.522962 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.254:37861
--webserver_interface=127.25.92.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab
--principal=kudu/127.25.92.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:40701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.92.254:37861
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:10:51.636193 1819 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:10:51.636536 1819 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:10:51.636622 1819 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:10:51.640789 1819 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:10:51.640897 1819 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:10:51.640934 1819 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:10:51.640996 1819 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:10:51.641050 1819 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:10:51.647399 1819 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:40701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.92.254:37861
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab
--principal=kudu/127.25.92.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.254:37861
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.92.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1819
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:10:51.648624 1819 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:10:51.649834 1819 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:10:51.658988 1827 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:51.659011 1825 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:51.659003 1824 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:10:51.659061 1819 server_base.cc:1034] running on GCE node
I20260805 23:10:51.659854 1819 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:10:51.660976 1819 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:10:51.662182 1819 hybrid_clock.cc:648] HybridClock initialized: now 1785971451662161 us; error 26 us; skew 500 ppm
Aug 05 23:10:51 dist-test-slave-ss4n krb5kdc[1806](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971451, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:10:51.665132 1819 init.cc:377] Logged in from keytab as kudu/127.25.92.254@KRBTEST.COM (short username kudu)
I20260805 23:10:51.666153 1819 webserver.cc:460] Webserver started at http://127.25.92.254:35375/ using document root <none> and password file <none>
I20260805 23:10:51.666802 1819 fs_manager.cc:359] Metadata directory not provided
I20260805 23:10:51.666909 1819 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:10:51.667146 1819 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:10:51.669304 1819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/master-0/data/instance:
uuid: "408aa6f5e15749f09e783ff2cb6f1276"
format_stamp: "Formatted at 2026-08-05 23:10:51 on dist-test-slave-ss4n"
server_key: "2d004a4216b51d1c17c4d2bd270d094a"
server_key_iv: "464a2dae231f95835b657ad451212760"
server_key_version: "encryptionkey@0"
I20260805 23:10:51.669942 1819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/master-0/wal/instance:
uuid: "408aa6f5e15749f09e783ff2cb6f1276"
format_stamp: "Formatted at 2026-08-05 23:10:51 on dist-test-slave-ss4n"
server_key: "2d004a4216b51d1c17c4d2bd270d094a"
server_key_iv: "464a2dae231f95835b657ad451212760"
server_key_version: "encryptionkey@0"
I20260805 23:10:51.673936 1819 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260805 23:10:51.676443 1835 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:51.677913 1819 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:10:51.678064 1819 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/master-0/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/master-0/wal
uuid: "408aa6f5e15749f09e783ff2cb6f1276"
format_stamp: "Formatted at 2026-08-05 23:10:51 on dist-test-slave-ss4n"
server_key: "2d004a4216b51d1c17c4d2bd270d094a"
server_key_iv: "464a2dae231f95835b657ad451212760"
server_key_version: "encryptionkey@0"
I20260805 23:10:51.678189 1819 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:10:51.703315 1819 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:10:51.706857 1819 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:10:51.707082 1819 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:10:51.715694 1819 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.254:37861
I20260805 23:10:51.715723 1891 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.254:37861 every 8 connection(s)
I20260805 23:10:51.716881 1819 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
I20260805 23:10:51.719726 1892 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:10:51.720920 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 1819
I20260805 23:10:51.721051 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/master-0/wal/instance
I20260805 23:10:51.721371 25971 external_mini_cluster.cc:1428] Setting key 072a60683c9f37363deef8970d272360
I20260805 23:10:51.726063 1892 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 408aa6f5e15749f09e783ff2cb6f1276: Bootstrap starting.
I20260805 23:10:51.728333 1892 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 408aa6f5e15749f09e783ff2cb6f1276: Neither blocks nor log segments found. Creating new log.
I20260805 23:10:51.729058 1892 log.cc:824] T 00000000000000000000000000000000 P 408aa6f5e15749f09e783ff2cb6f1276: Log is configured to *not* fsync() on all Append() calls
Aug 05 23:10:51 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971451, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM
I20260805 23:10:51.731159 1892 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 408aa6f5e15749f09e783ff2cb6f1276: No bootstrap required, opened a new log
I20260805 23:10:51.734584 1892 raft_consensus.cc:348] T 00000000000000000000000000000000 P 408aa6f5e15749f09e783ff2cb6f1276 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "408aa6f5e15749f09e783ff2cb6f1276" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 37861 } }
I20260805 23:10:51.734803 1892 raft_consensus.cc:374] T 00000000000000000000000000000000 P 408aa6f5e15749f09e783ff2cb6f1276 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:10:51.734861 1892 raft_consensus.cc:729] T 00000000000000000000000000000000 P 408aa6f5e15749f09e783ff2cb6f1276 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 408aa6f5e15749f09e783ff2cb6f1276, State: Initialized, Role: FOLLOWER
I20260805 23:10:51.735329 1892 consensus_queue.cc:260] T 00000000000000000000000000000000 P 408aa6f5e15749f09e783ff2cb6f1276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "408aa6f5e15749f09e783ff2cb6f1276" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 37861 } }
I20260805 23:10:51.735484 1892 raft_consensus.cc:388] T 00000000000000000000000000000000 P 408aa6f5e15749f09e783ff2cb6f1276 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:10:51.735589 1892 raft_consensus.cc:482] T 00000000000000000000000000000000 P 408aa6f5e15749f09e783ff2cb6f1276 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:10:51.735699 1892 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 408aa6f5e15749f09e783ff2cb6f1276 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:10:51.736725 1892 raft_consensus.cc:504] T 00000000000000000000000000000000 P 408aa6f5e15749f09e783ff2cb6f1276 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "408aa6f5e15749f09e783ff2cb6f1276" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 37861 } }
I20260805 23:10:51.737073 1892 leader_election.cc:302] T 00000000000000000000000000000000 P 408aa6f5e15749f09e783ff2cb6f1276 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 408aa6f5e15749f09e783ff2cb6f1276; no voters:
I20260805 23:10:51.737109 1895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:51.722943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34356 (local address 127.25.92.254:37861)
0805 23:10:51.723420 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:51.723436 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:51.723481 (+ 45us) server_negotiation.cc:407] Connection header received
0805 23:10:51.724184 (+ 703us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:51.724202 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:51.724538 (+ 336us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:51.724894 (+ 356us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:51.725852 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:51.727012 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:51.728084 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:51.728665 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:51.728842 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:51.731861 (+ 3019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:51.731866 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:51.731881 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:51.731912 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:51.733935 (+ 2023us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:51.734429 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:51.734441 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:51.734446 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:51.734563 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:51.734927 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:51.734931 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:51.734933 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:51.735358 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:51.735600 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:51.735908 (+ 308us) server_negotiation.cc:299] Negotiation successful
0805 23:10:51.736256 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326,"thread_start_us":139,"threads_started":1}
I20260805 23:10:51.737434 1892 leader_election.cc:288] T 00000000000000000000000000000000 P 408aa6f5e15749f09e783ff2cb6f1276 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 23:10:51.737648 1897 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 408aa6f5e15749f09e783ff2cb6f1276 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:10:51.737974 1897 raft_consensus.cc:686] T 00000000000000000000000000000000 P 408aa6f5e15749f09e783ff2cb6f1276 [term 1 LEADER]: Becoming Leader. State: Replica: 408aa6f5e15749f09e783ff2cb6f1276, State: Running, Role: LEADER
I20260805 23:10:51.738350 1897 consensus_queue.cc:237] T 00000000000000000000000000000000 P 408aa6f5e15749f09e783ff2cb6f1276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "408aa6f5e15749f09e783ff2cb6f1276" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 37861 } }
I20260805 23:10:51.738757 1892 sys_catalog.cc:564] T 00000000000000000000000000000000 P 408aa6f5e15749f09e783ff2cb6f1276 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:10:51.740123 1898 sys_catalog.cc:455] T 00000000000000000000000000000000 P 408aa6f5e15749f09e783ff2cb6f1276 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "408aa6f5e15749f09e783ff2cb6f1276" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "408aa6f5e15749f09e783ff2cb6f1276" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 37861 } } }
I20260805 23:10:51.740252 1898 sys_catalog.cc:458] T 00000000000000000000000000000000 P 408aa6f5e15749f09e783ff2cb6f1276 [sys.catalog]: This master's current role is: LEADER
I20260805 23:10:51.740614 1899 sys_catalog.cc:455] T 00000000000000000000000000000000 P 408aa6f5e15749f09e783ff2cb6f1276 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 408aa6f5e15749f09e783ff2cb6f1276. Latest consensus state: current_term: 1 leader_uuid: "408aa6f5e15749f09e783ff2cb6f1276" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "408aa6f5e15749f09e783ff2cb6f1276" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 37861 } } }
I20260805 23:10:51.740767 1899 sys_catalog.cc:458] T 00000000000000000000000000000000 P 408aa6f5e15749f09e783ff2cb6f1276 [sys.catalog]: This master's current role is: LEADER
I20260805 23:10:51.740680 1906 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:10:51.743634 1906 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:10:51.748971 1906 catalog_manager.cc:1347] Generated new cluster ID: a8859c69c85d47a2bc06728dad4ad3c2
I20260805 23:10:51.749061 1906 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:10:51.769111 1906 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:10:51.770292 1906 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:10:51.778170 1906 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 408aa6f5e15749f09e783ff2cb6f1276: Generated new TSK 0
I20260805 23:10:51.778972 1906 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.92.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.92.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.92.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.193.keytab.
Entry for principal kudu/127.25.92.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.193.keytab.
I20260805 23:10:51.817902 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.92.193:0
--local_ip_for_outbound_sockets=127.25.92.193
--webserver_interface=127.25.92.193
--webserver_port=0
--tserver_master_addrs=127.25.92.254:37861
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.193.keytab
--principal=kudu/127.25.92.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:40701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:10:51.938263 1918 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:10:51.938596 1918 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:10:51.938685 1918 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:10:51.943370 1918 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:10:51.943467 1918 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:10:51.943562 1918 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.92.193
I20260805 23:10:51.948971 1918 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:40701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.193.keytab
--principal=kudu/127.25.92.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.92.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.92.193
--webserver_port=0
--tserver_master_addrs=127.25.92.254:37861
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1918
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.92.193
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:10:51.950196 1918 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:10:51.951248 1918 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:10:51.961411 1926 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:51.961616 1923 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:10:51.961637 1918 server_base.cc:1034] running on GCE node
W20260805 23:10:51.961418 1924 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:10:51.962144 1918 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:10:51.962893 1918 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:10:51.964092 1918 hybrid_clock.cc:648] HybridClock initialized: now 1785971451964053 us; error 51 us; skew 500 ppm
Aug 05 23:10:51 dist-test-slave-ss4n krb5kdc[1806](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971451, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:10:51.967132 1918 init.cc:377] Logged in from keytab as kudu/127.25.92.193@KRBTEST.COM (short username kudu)
I20260805 23:10:51.967972 1918 webserver.cc:460] Webserver started at http://127.25.92.193:43875/ using document root <none> and password file <none>
I20260805 23:10:51.968502 1918 fs_manager.cc:359] Metadata directory not provided
I20260805 23:10:51.968562 1918 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:10:51.968726 1918 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:10:51.970796 1918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/ts-0/data/instance:
uuid: "960fbc0fa34a4e8784df458925c52b6a"
format_stamp: "Formatted at 2026-08-05 23:10:51 on dist-test-slave-ss4n"
server_key: "d804b060418efa3f8aca5488c8929a85"
server_key_iv: "dfd62512ca050ecfe7141911de9b4b51"
server_key_version: "encryptionkey@0"
I20260805 23:10:51.971340 1918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/ts-0/wal/instance:
uuid: "960fbc0fa34a4e8784df458925c52b6a"
format_stamp: "Formatted at 2026-08-05 23:10:51 on dist-test-slave-ss4n"
server_key: "d804b060418efa3f8aca5488c8929a85"
server_key_iv: "dfd62512ca050ecfe7141911de9b4b51"
server_key_version: "encryptionkey@0"
I20260805 23:10:51.975143 1918 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260805 23:10:51.977661 1933 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:51.979038 1918 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:10:51.979180 1918 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/ts-0/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/ts-0/wal
uuid: "960fbc0fa34a4e8784df458925c52b6a"
format_stamp: "Formatted at 2026-08-05 23:10:51 on dist-test-slave-ss4n"
server_key: "d804b060418efa3f8aca5488c8929a85"
server_key_iv: "dfd62512ca050ecfe7141911de9b4b51"
server_key_version: "encryptionkey@0"
I20260805 23:10:51.979307 1918 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:10:51.991205 1918 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:10:51.994542 1918 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:10:51.994745 1918 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:10:51.995328 1918 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:10:51.996170 1918 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:10:51.996243 1918 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:51.996309 1918 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:10:51.996352 1918 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:52.005339 1918 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.193:39737
I20260805 23:10:52.005366 2051 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.193:39737 every 8 connection(s)
I20260805 23:10:52.006279 1918 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/ts-0/data/info.pb
I20260805 23:10:52.015656 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 1918
I20260805 23:10:52.015803 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/ts-0/wal/instance
I20260805 23:10:52.016129 25971 external_mini_cluster.cc:1428] Setting key f22e9a4a6ba4d015a0e07ea2e2b8b0af
Aug 05 23:10:52 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971451, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.193@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM
I20260805 23:10:52.021677 1895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:52.008171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.193:41427 (local address 127.25.92.254:37861)
0805 23:10:52.008364 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:52.008369 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:52.009272 (+ 903us) server_negotiation.cc:407] Connection header received
0805 23:10:52.010672 (+ 1400us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:52.010675 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:52.010724 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:52.010803 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:52.011990 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:52.012602 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:52.013827 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:52.014115 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:52.014220 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:52.017487 (+ 3267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:52.017518 (+ 31us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:52.017524 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:52.017556 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:52.019096 (+ 1540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:52.019750 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:52.019756 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:52.019759 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:52.019861 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:52.020243 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:52.020249 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:52.020251 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:52.020512 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:52.020624 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:52.021208 (+ 584us) server_negotiation.cc:299] Negotiation successful
0805 23:10:52.021379 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260805 23:10:52.022478 2054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:52.008659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:37861 (local address 127.25.92.193:41427)
0805 23:10:52.009114 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:52.009158 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:52.010428 (+ 1270us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:52.010966 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:52.010973 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:52.011323 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:52.011822 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:52.011837 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:52.012729 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:52.012732 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:52.013704 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:52.013715 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:52.014262 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:52.014265 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:52.014402 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:52.015131 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:10:52.015147 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:52.017273 (+ 2126us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:10:52.019231 (+ 1958us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:52.019242 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:52.019263 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:52.019640 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:52.019968 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:52.019970 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:52.019972 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:52.020140 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:52.020634 (+ 494us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:10:52.020639 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:10:52.020922 (+ 283us) client_negotiation.cc:769] Sending connection context
0805 23:10:52.021113 (+ 191us) client_negotiation.cc:240] Negotiation successful
0805 23:10:52.021401 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":114,"threads_started":1}
I20260805 23:10:52.023994 2052 heartbeater.cc:347] Connected to a master server at 127.25.92.254:37861
I20260805 23:10:52.024261 2052 heartbeater.cc:464] Registering TS with master...
I20260805 23:10:52.024847 2052 heartbeater.cc:511] Master 127.25.92.254:37861 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.25.92.194@KRBTEST.COM; defaulting to no policy
I20260805 23:10:52.026564 1852 ts_manager.cc:194] Registered new tserver with Master: 960fbc0fa34a4e8784df458925c52b6a (127.25.92.193:39737)
I20260805 23:10:52.027885 1852 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.92.193@KRBTEST.COM'} at 127.25.92.193:41427
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.92.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.92.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.194.keytab.
Entry for principal kudu/127.25.92.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.194.keytab.
I20260805 23:10:52.049021 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.92.194:0
--local_ip_for_outbound_sockets=127.25.92.194
--webserver_interface=127.25.92.194
--webserver_port=0
--tserver_master_addrs=127.25.92.254:37861
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.194.keytab
--principal=kudu/127.25.92.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:40701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:10:52.163425 2057 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:10:52.163707 2057 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:10:52.163753 2057 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:10:52.167865 2057 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:10:52.167927 2057 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:10:52.167990 2057 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.92.194
I20260805 23:10:52.173234 2057 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:40701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.194.keytab
--principal=kudu/127.25.92.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.92.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.92.194
--webserver_port=0
--tserver_master_addrs=127.25.92.254:37861
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2057
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.92.194
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:10:52.174353 2057 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:10:52.175417 2057 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:10:52.185251 2065 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:52.185251 2062 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:10:52.185417 2057 server_base.cc:1034] running on GCE node
W20260805 23:10:52.185742 2063 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:10:52.186208 2057 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:10:52.186812 2057 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:10:52.188059 2057 hybrid_clock.cc:648] HybridClock initialized: now 1785971452188040 us; error 31 us; skew 500 ppm
Aug 05 23:10:52 dist-test-slave-ss4n krb5kdc[1806](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971452, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:10:52.191222 2057 init.cc:377] Logged in from keytab as kudu/127.25.92.194@KRBTEST.COM (short username kudu)
I20260805 23:10:52.192184 2057 webserver.cc:460] Webserver started at http://127.25.92.194:41533/ using document root <none> and password file <none>
I20260805 23:10:52.192767 2057 fs_manager.cc:359] Metadata directory not provided
I20260805 23:10:52.192862 2057 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:10:52.193104 2057 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:10:52.195219 2057 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/ts-1/data/instance:
uuid: "42e2c18a010b42aea0b5d893b5191e9d"
format_stamp: "Formatted at 2026-08-05 23:10:52 on dist-test-slave-ss4n"
server_key: "0ba22d9aa7a3d19b4c30f60e2a042398"
server_key_iv: "a1aaa446451f563188bd0bf960724d43"
server_key_version: "encryptionkey@0"
I20260805 23:10:52.195868 2057 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/ts-1/wal/instance:
uuid: "42e2c18a010b42aea0b5d893b5191e9d"
format_stamp: "Formatted at 2026-08-05 23:10:52 on dist-test-slave-ss4n"
server_key: "0ba22d9aa7a3d19b4c30f60e2a042398"
server_key_iv: "a1aaa446451f563188bd0bf960724d43"
server_key_version: "encryptionkey@0"
I20260805 23:10:52.199826 2057 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260805 23:10:52.202658 2073 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:52.204033 2057 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:10:52.204200 2057 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/ts-1/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/ts-1/wal
uuid: "42e2c18a010b42aea0b5d893b5191e9d"
format_stamp: "Formatted at 2026-08-05 23:10:52 on dist-test-slave-ss4n"
server_key: "0ba22d9aa7a3d19b4c30f60e2a042398"
server_key_iv: "a1aaa446451f563188bd0bf960724d43"
server_key_version: "encryptionkey@0"
I20260805 23:10:52.204327 2057 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:10:52.213775 2057 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:10:52.217252 2057 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:10:52.217465 2057 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:10:52.218151 2057 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:10:52.219012 2057 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:10:52.219080 2057 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:52.219153 2057 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:10:52.219206 2057 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:52.229051 2057 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.194:39391
I20260805 23:10:52.229070 2190 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.194:39391 every 8 connection(s)
I20260805 23:10:52.230104 2057 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/ts-1/data/info.pb
I20260805 23:10:52.236373 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 2057
I20260805 23:10:52.236518 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/ts-1/wal/instance
I20260805 23:10:52.236840 25971 external_mini_cluster.cc:1428] Setting key 218807b08d89fbb1661adc24002e09b2
Aug 05 23:10:52 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971452, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.194@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM
I20260805 23:10:52.247176 1895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:52.232076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.194:56141 (local address 127.25.92.254:37861)
0805 23:10:52.232232 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:52.232237 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:52.233185 (+ 948us) server_negotiation.cc:407] Connection header received
0805 23:10:52.234668 (+ 1483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:52.234671 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:52.234722 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:52.234799 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:52.236162 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:52.237027 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:52.238857 (+ 1830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:52.239277 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:52.239414 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:52.242701 (+ 3287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:52.242709 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:52.242714 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:52.242751 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:52.244487 (+ 1736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:52.245258 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:52.245261 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:52.245262 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:52.245347 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:52.245833 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:52.245836 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:52.245837 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:52.246048 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:52.246129 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:52.246907 (+ 778us) server_negotiation.cc:299] Negotiation successful
0805 23:10:52.247046 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
WARNING: no policy specified for kudu/127.25.92.195@KRBTEST.COM; defaulting to no policy
I20260805 23:10:52.248442 2193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:52.232520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:37861 (local address 127.25.92.194:56141)
0805 23:10:52.233025 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:52.233072 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:52.234431 (+ 1359us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:52.234966 (+ 535us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:52.234973 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:52.235341 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:52.235901 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:52.235915 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:52.237167 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:52.237169 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:52.238695 (+ 1526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:52.238709 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:52.239463 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:52.239468 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:52.239692 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:52.240751 (+ 1059us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:10:52.240784 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:52.242491 (+ 1707us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:10:52.244629 (+ 2138us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:52.244639 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:52.244670 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:52.245123 (+ 453us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:52.245454 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:52.245457 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:52.245460 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:52.245714 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:52.246167 (+ 453us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:10:52.246175 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:10:52.246561 (+ 386us) client_negotiation.cc:769] Sending connection context
0805 23:10:52.246836 (+ 275us) client_negotiation.cc:240] Negotiation successful
0805 23:10:52.247257 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":123,"threads_started":1}
I20260805 23:10:52.249936 2191 heartbeater.cc:347] Connected to a master server at 127.25.92.254:37861
I20260805 23:10:52.250173 2191 heartbeater.cc:464] Registering TS with master...
I20260805 23:10:52.250780 2191 heartbeater.cc:511] Master 127.25.92.254:37861 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.92.195@KRBTEST.COM" created.
I20260805 23:10:52.252105 1852 ts_manager.cc:194] Registered new tserver with Master: 42e2c18a010b42aea0b5d893b5191e9d (127.25.92.194:39391)
I20260805 23:10:52.252806 1852 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.92.194@KRBTEST.COM'} at 127.25.92.194:56141
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.92.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.195.keytab.
Entry for principal kudu/127.25.92.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.195.keytab.
I20260805 23:10:52.269333 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.92.195:0
--local_ip_for_outbound_sockets=127.25.92.195
--webserver_interface=127.25.92.195
--webserver_port=0
--tserver_master_addrs=127.25.92.254:37861
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.195.keytab
--principal=kudu/127.25.92.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:40701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:10:52.384550 2197 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:10:52.384814 2197 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:10:52.384859 2197 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:10:52.389050 2197 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:10:52.389112 2197 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:10:52.389216 2197 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.92.195
I20260805 23:10:52.394557 2197 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:40701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.195.keytab
--principal=kudu/127.25.92.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.92.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.92.195
--webserver_port=0
--tserver_master_addrs=127.25.92.254:37861
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2197
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.92.195
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:10:52.395664 2197 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:10:52.396704 2197 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:10:52.406513 2205 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:52.406615 2203 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:52.406533 2202 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:10:52.406736 2197 server_base.cc:1034] running on GCE node
I20260805 23:10:52.407454 2197 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:10:52.408217 2197 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:10:52.409422 2197 hybrid_clock.cc:648] HybridClock initialized: now 1785971452409385 us; error 57 us; skew 500 ppm
Aug 05 23:10:52 dist-test-slave-ss4n krb5kdc[1806](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971452, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:10:52.412473 2197 init.cc:377] Logged in from keytab as kudu/127.25.92.195@KRBTEST.COM (short username kudu)
I20260805 23:10:52.413743 2197 webserver.cc:460] Webserver started at http://127.25.92.195:35087/ using document root <none> and password file <none>
I20260805 23:10:52.414350 2197 fs_manager.cc:359] Metadata directory not provided
I20260805 23:10:52.414450 2197 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:10:52.414700 2197 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:10:52.417141 2197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/ts-2/data/instance:
uuid: "7004e2e62537442aac2e12a8ab98d501"
format_stamp: "Formatted at 2026-08-05 23:10:52 on dist-test-slave-ss4n"
server_key: "28b9036997c580e2ce2e7084ac8880cb"
server_key_iv: "856e63828b557c86f6b7aaf92f704e37"
server_key_version: "encryptionkey@0"
I20260805 23:10:52.417850 2197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/ts-2/wal/instance:
uuid: "7004e2e62537442aac2e12a8ab98d501"
format_stamp: "Formatted at 2026-08-05 23:10:52 on dist-test-slave-ss4n"
server_key: "28b9036997c580e2ce2e7084ac8880cb"
server_key_iv: "856e63828b557c86f6b7aaf92f704e37"
server_key_version: "encryptionkey@0"
I20260805 23:10:52.422230 2197 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260805 23:10:52.425446 2212 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:52.426913 2197 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:10:52.427089 2197 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/ts-2/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/ts-2/wal
uuid: "7004e2e62537442aac2e12a8ab98d501"
format_stamp: "Formatted at 2026-08-05 23:10:52 on dist-test-slave-ss4n"
server_key: "28b9036997c580e2ce2e7084ac8880cb"
server_key_iv: "856e63828b557c86f6b7aaf92f704e37"
server_key_version: "encryptionkey@0"
I20260805 23:10:52.427251 2197 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:10:52.452390 2197 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:10:52.456171 2197 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:10:52.456429 2197 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:10:52.457101 2197 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:10:52.458169 2197 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:10:52.458249 2197 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:52.458330 2197 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:10:52.458379 2197 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:52.468051 2197 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.195:40521
I20260805 23:10:52.468140 2329 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.195:40521 every 8 connection(s)
I20260805 23:10:52.469069 2197 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/ts-2/data/info.pb
I20260805 23:10:52.478222 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 2197
I20260805 23:10:52.478402 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/ts-2/wal/instance
I20260805 23:10:52.478776 25971 external_mini_cluster.cc:1428] Setting key 02932943bdefaac8e4045aae86a2aae1
Aug 05 23:10:52 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971452, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.195@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM
I20260805 23:10:52.484954 1895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:52.471235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.195:34387 (local address 127.25.92.254:37861)
0805 23:10:52.471426 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:52.471432 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:52.472243 (+ 811us) server_negotiation.cc:407] Connection header received
0805 23:10:52.473684 (+ 1441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:52.473687 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:52.473742 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:52.473826 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:52.475067 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:52.475685 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:52.476951 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:52.477316 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:52.477414 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:52.480617 (+ 3203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:52.480621 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:52.480625 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:52.480654 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:52.482497 (+ 1843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:52.483044 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:52.483046 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:52.483048 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:52.483296 (+ 248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:52.483675 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:52.483678 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:52.483679 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:52.483886 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:52.483976 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:52.484720 (+ 744us) server_negotiation.cc:299] Negotiation successful
0805 23:10:52.484831 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260805 23:10:52.486083 2332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:52.471583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:37861 (local address 127.25.92.195:34387)
0805 23:10:52.472081 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:52.472125 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:52.473352 (+ 1227us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:52.473997 (+ 645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:52.474005 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:52.474358 (+ 353us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:52.474888 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:52.474902 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:52.475823 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:52.475826 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:52.476807 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:52.476816 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:52.477622 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:52.477628 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:52.477790 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:52.478544 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:10:52.478566 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:52.480360 (+ 1794us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:10:52.482621 (+ 2261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:52.482626 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:52.482640 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:52.482931 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:52.483385 (+ 454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:10:52.483387 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:10:52.483389 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:10:52.483552 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:10:52.484022 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:10:52.484029 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:10:52.484415 (+ 386us) client_negotiation.cc:769] Sending connection context
0805 23:10:52.484647 (+ 232us) client_negotiation.cc:240] Negotiation successful
0805 23:10:52.484982 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":113,"threads_started":1}
I20260805 23:10:52.487402 2330 heartbeater.cc:347] Connected to a master server at 127.25.92.254:37861
I20260805 23:10:52.487732 2330 heartbeater.cc:464] Registering TS with master...
I20260805 23:10:52.488282 2330 heartbeater.cc:511] Master 127.25.92.254:37861 requested a full tablet report, sending...
I20260805 23:10:52.489418 1852 ts_manager.cc:194] Registered new tserver with Master: 7004e2e62537442aac2e12a8ab98d501 (127.25.92.195:40521)
I20260805 23:10:52.490152 1852 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.92.195@KRBTEST.COM'} at 127.25.92.195:34387
I20260805 23:10:52.493175 25971 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.25.92.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.25.92.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.25.92.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/krb5kdc/oryx_127.25.92.193.keytab.
Entry for principal oryx/127.25.92.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/krb5kdc/oryx_127.25.92.193.keytab.
WARNING: no policy specified for oryx/127.25.92.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.25.92.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.25.92.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/krb5kdc/oryx_127.25.92.194.keytab.
Entry for principal oryx/127.25.92.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/krb5kdc/oryx_127.25.92.194.keytab.
WARNING: no policy specified for oryx/127.25.92.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.25.92.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.25.92.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/krb5kdc/oryx_127.25.92.195.keytab.
Entry for principal oryx/127.25.92.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/krb5kdc/oryx_127.25.92.195.keytab.
I20260805 23:10:52.581233 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 1918
I20260805 23:10:52.587723 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 2057
I20260805 23:10:52.593927 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 2197
I20260805 23:10:52.600184 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 1819
I20260805 23:10:52.607630 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.254:37861
--webserver_interface=127.25.92.254
--webserver_port=35375
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab
--principal=kudu/127.25.92.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:40701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.92.254:37861
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:10:52.727478 2339 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:10:52.727797 2339 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:10:52.727840 2339 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:10:52.732074 2339 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:10:52.732149 2339 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:10:52.732170 2339 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:10:52.732187 2339 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:10:52.732203 2339 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:10:52.738195 2339 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:40701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.92.254:37861
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab
--principal=kudu/127.25.92.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.254:37861
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.92.254
--webserver_port=35375
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2339
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:10:52.739382 2339 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:10:52.740433 2339 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:10:52.750061 2347 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:52.750061 2344 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:10:52.750138 2339 server_base.cc:1034] running on GCE node
W20260805 23:10:52.750070 2345 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:10:52.750970 2339 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:10:52.752158 2339 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:10:52.753355 2339 hybrid_clock.cc:648] HybridClock initialized: now 1785971452753318 us; error 50 us; skew 500 ppm
Aug 05 23:10:52 dist-test-slave-ss4n krb5kdc[1806](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971452, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:10:52.756623 2339 init.cc:377] Logged in from keytab as kudu/127.25.92.254@KRBTEST.COM (short username kudu)
I20260805 23:10:52.757740 2339 webserver.cc:460] Webserver started at http://127.25.92.254:35375/ using document root <none> and password file <none>
I20260805 23:10:52.758401 2339 fs_manager.cc:359] Metadata directory not provided
I20260805 23:10:52.758533 2339 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:10:52.763007 2339 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.003s
I20260805 23:10:52.765532 2354 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:52.766942 2339 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260805 23:10:52.767115 2339 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/master-0/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/master-0/wal
uuid: "408aa6f5e15749f09e783ff2cb6f1276"
format_stamp: "Formatted at 2026-08-05 23:10:51 on dist-test-slave-ss4n"
server_key: "2d004a4216b51d1c17c4d2bd270d094a"
server_key_iv: "464a2dae231f95835b657ad451212760"
server_key_version: "encryptionkey@0"
I20260805 23:10:52.767753 2339 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:10:52.782609 2339 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:10:52.786369 2339 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:10:52.786599 2339 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:10:52.794389 2339 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.254:37861
I20260805 23:10:52.794404 2407 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.254:37861 every 8 connection(s)
I20260805 23:10:52.795550 2339 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
I20260805 23:10:52.800359 2409 sys_catalog.cc:263] Verifying existing consensus state
I20260805 23:10:52.802284 2409 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 408aa6f5e15749f09e783ff2cb6f1276: Bootstrap starting.
I20260805 23:10:52.805408 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 2339
I20260805 23:10:52.805894 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.92.193:39737
--local_ip_for_outbound_sockets=127.25.92.193
--tserver_master_addrs=127.25.92.254:37861
--webserver_port=43875
--webserver_interface=127.25.92.193
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.193.keytab
--principal=kudu/127.25.92.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:40701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.25.92.193
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/krb5kdc/oryx_127.25.92.193.keytab with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260805 23:10:52.821214 2409 log.cc:824] T 00000000000000000000000000000000 P 408aa6f5e15749f09e783ff2cb6f1276: Log is configured to *not* fsync() on all Append() calls
I20260805 23:10:52.829218 2409 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 408aa6f5e15749f09e783ff2cb6f1276: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260805 23:10:52.829718 2409 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 408aa6f5e15749f09e783ff2cb6f1276: Bootstrap complete.
I20260805 23:10:52.834465 2409 raft_consensus.cc:348] T 00000000000000000000000000000000 P 408aa6f5e15749f09e783ff2cb6f1276 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "408aa6f5e15749f09e783ff2cb6f1276" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 37861 } }
I20260805 23:10:52.834966 2409 raft_consensus.cc:729] T 00000000000000000000000000000000 P 408aa6f5e15749f09e783ff2cb6f1276 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 408aa6f5e15749f09e783ff2cb6f1276, State: Initialized, Role: FOLLOWER
I20260805 23:10:52.835593 2409 consensus_queue.cc:260] T 00000000000000000000000000000000 P 408aa6f5e15749f09e783ff2cb6f1276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "408aa6f5e15749f09e783ff2cb6f1276" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 37861 } }
I20260805 23:10:52.835723 2409 raft_consensus.cc:388] T 00000000000000000000000000000000 P 408aa6f5e15749f09e783ff2cb6f1276 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:10:52.835822 2409 raft_consensus.cc:482] T 00000000000000000000000000000000 P 408aa6f5e15749f09e783ff2cb6f1276 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:10:52.835934 2409 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 408aa6f5e15749f09e783ff2cb6f1276 [term 1 FOLLOWER]: Advancing to term 2
I20260805 23:10:52.838253 2409 raft_consensus.cc:504] T 00000000000000000000000000000000 P 408aa6f5e15749f09e783ff2cb6f1276 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "408aa6f5e15749f09e783ff2cb6f1276" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 37861 } }
I20260805 23:10:52.838635 2409 leader_election.cc:302] T 00000000000000000000000000000000 P 408aa6f5e15749f09e783ff2cb6f1276 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 408aa6f5e15749f09e783ff2cb6f1276; no voters:
I20260805 23:10:52.839025 2409 leader_election.cc:288] T 00000000000000000000000000000000 P 408aa6f5e15749f09e783ff2cb6f1276 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260805 23:10:52.839178 2413 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 408aa6f5e15749f09e783ff2cb6f1276 [term 2 FOLLOWER]: Leader election won for term 2
I20260805 23:10:52.839452 2413 raft_consensus.cc:686] T 00000000000000000000000000000000 P 408aa6f5e15749f09e783ff2cb6f1276 [term 2 LEADER]: Becoming Leader. State: Replica: 408aa6f5e15749f09e783ff2cb6f1276, State: Running, Role: LEADER
I20260805 23:10:52.839859 2413 consensus_queue.cc:237] T 00000000000000000000000000000000 P 408aa6f5e15749f09e783ff2cb6f1276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "408aa6f5e15749f09e783ff2cb6f1276" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 37861 } }
I20260805 23:10:52.840445 2409 sys_catalog.cc:564] T 00000000000000000000000000000000 P 408aa6f5e15749f09e783ff2cb6f1276 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:10:52.841434 2415 sys_catalog.cc:455] T 00000000000000000000000000000000 P 408aa6f5e15749f09e783ff2cb6f1276 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "408aa6f5e15749f09e783ff2cb6f1276" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "408aa6f5e15749f09e783ff2cb6f1276" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 37861 } } }
I20260805 23:10:52.841594 2415 sys_catalog.cc:458] T 00000000000000000000000000000000 P 408aa6f5e15749f09e783ff2cb6f1276 [sys.catalog]: This master's current role is: LEADER
I20260805 23:10:52.842481 2416 sys_catalog.cc:455] T 00000000000000000000000000000000 P 408aa6f5e15749f09e783ff2cb6f1276 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 408aa6f5e15749f09e783ff2cb6f1276. Latest consensus state: current_term: 2 leader_uuid: "408aa6f5e15749f09e783ff2cb6f1276" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "408aa6f5e15749f09e783ff2cb6f1276" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 37861 } } }
I20260805 23:10:52.842594 2416 sys_catalog.cc:458] T 00000000000000000000000000000000 P 408aa6f5e15749f09e783ff2cb6f1276 [sys.catalog]: This master's current role is: LEADER
I20260805 23:10:52.842877 2423 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:10:52.846731 2423 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:10:52.847361 2423 catalog_manager.cc:1259] Loaded cluster ID: a8859c69c85d47a2bc06728dad4ad3c2
I20260805 23:10:52.847422 2423 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:10:52.849828 2423 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:10:52.850495 2423 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 408aa6f5e15749f09e783ff2cb6f1276: Loaded TSK: 0
I20260805 23:10:52.851714 2423 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260805 23:10:52.929808 2411 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:10:52.930068 2411 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:10:52.930114 2411 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:10:52.934293 2411 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:10:52.934357 2411 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:10:52.934418 2411 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.92.193
I20260805 23:10:52.939945 2411 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:40701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/krb5kdc/oryx_127.25.92.193.keytab
--principal=oryx/127.25.92.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.92.193:39737
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.92.193
--webserver_port=43875
--tserver_master_addrs=127.25.92.254:37861
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2411
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.92.193
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:10:52.941135 2411 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:10:52.942358 2411 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:10:52.953876 2437 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:52.953869 2436 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:10:52.954397 2411 server_base.cc:1034] running on GCE node
W20260805 23:10:52.954469 2439 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:10:52.954995 2411 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:10:52.955796 2411 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:10:52.957124 2411 hybrid_clock.cc:648] HybridClock initialized: now 1785971452957113 us; error 49 us; skew 500 ppm
Aug 05 23:10:52 dist-test-slave-ss4n krb5kdc[1806](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971452, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.92.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:10:52.960372 2411 init.cc:377] Logged in from keytab as oryx/127.25.92.193@KRBTEST.COM (short username oryx)
I20260805 23:10:52.961630 2411 webserver.cc:460] Webserver started at http://127.25.92.193:43875/ using document root <none> and password file <none>
I20260805 23:10:52.962327 2411 fs_manager.cc:359] Metadata directory not provided
I20260805 23:10:52.962468 2411 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:10:52.967242 2411 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260805 23:10:52.969771 2447 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:52.971290 2411 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260805 23:10:52.971452 2411 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/ts-0/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/ts-0/wal
uuid: "960fbc0fa34a4e8784df458925c52b6a"
format_stamp: "Formatted at 2026-08-05 23:10:51 on dist-test-slave-ss4n"
server_key: "d804b060418efa3f8aca5488c8929a85"
server_key_iv: "dfd62512ca050ecfe7141911de9b4b51"
server_key_version: "encryptionkey@0"
I20260805 23:10:52.972074 2411 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:10:52.989336 2411 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:10:52.992762 2411 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:10:52.993021 2411 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:10:52.993726 2411 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:10:52.994663 2411 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:10:52.994724 2411 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:52.994804 2411 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:10:52.994838 2411 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:53.005208 2411 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.193:39737
I20260805 23:10:53.005232 2565 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.193:39737 every 8 connection(s)
I20260805 23:10:53.006383 2411 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/ts-0/data/info.pb
I20260805 23:10:53.013404 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 2411
I20260805 23:10:53.014068 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.92.194:39391
--local_ip_for_outbound_sockets=127.25.92.194
--tserver_master_addrs=127.25.92.254:37861
--webserver_port=41533
--webserver_interface=127.25.92.194
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.194.keytab
--principal=kudu/127.25.92.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:40701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.25.92.194
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/krb5kdc/oryx_127.25.92.194.keytab with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 05 23:10:53 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.193@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:10:53 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.193@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
I20260805 23:10:53.021265 2569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:53.008679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:37861 (local address 127.25.92.193:48343)
0805 23:10:53.009195 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:53.009238 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:53.010535 (+ 1297us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:53.011848 (+ 1313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:53.011855 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:53.012260 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:53.012767 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:53.012782 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:53.014349 (+ 1567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:53.014352 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:53.015235 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:53.015245 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:53.016524 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:53.016528 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:53.016740 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:53.017447 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:10:53.017475 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:53.020441 (+ 2966us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.92.254:37861: Server oryx/127.25.92.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":159,"threads_started":1}
W20260805 23:10:53.021826 2568 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:10:53.008572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.193:48343 (local address 127.25.92.254:37861)
0805 23:10:53.009134 (+ 562us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:53.009156 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:53.009355 (+ 199us) server_negotiation.cc:407] Connection header received
0805 23:10:53.010955 (+ 1600us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:53.010967 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:53.011281 (+ 314us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:53.011583 (+ 302us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:53.012960 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:53.014120 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:53.015380 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:53.016331 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:53.016553 (+ 222us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:53.020766 (+ 4213us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.92.193:48343: BlockingRecv error: recv got EOF from 127.25.92.193:48343 (error 108)
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":120,"threads_started":1}
W20260805 23:10:53.021987 2566 heartbeater.cc:644] Failed to heartbeat to 127.25.92.254:37861 (0 consecutive failures): Not authorized: Failed to ping master at 127.25.92.254:37861: Client connection negotiation failed: client connection to 127.25.92.254:37861: Server oryx/127.25.92.254@KRBTEST.COM not found in Kerberos database
Aug 05 23:10:53 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.193@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:10:53 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.193@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
I20260805 23:10:53.028517 2569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:53.022538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:37861 (local address 127.25.92.193:42389)
0805 23:10:53.022767 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:53.022785 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:53.022887 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:53.023213 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:53.023217 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:53.023408 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:53.023518 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:53.023525 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:53.024367 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:53.024374 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:53.024999 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:53.025008 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:53.025562 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:53.025567 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:53.025680 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:53.026304 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:10:53.026318 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:53.028353 (+ 2035us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.92.254:37861: Server oryx/127.25.92.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114}
W20260805 23:10:53.028836 2568 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:10:53.022659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.193:42389 (local address 127.25.92.254:37861)
0805 23:10:53.022818 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:53.022826 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:53.022846 (+ 20us) server_negotiation.cc:407] Connection header received
0805 23:10:53.023023 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:53.023027 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:53.023087 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:53.023197 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:53.023652 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:53.024236 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:53.025138 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:53.025398 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:53.025567 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:53.028708 (+ 3141us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.92.193:42389: BlockingRecv error: recv got EOF from 127.25.92.193:42389 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 05 23:10:53 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.193@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:10:53 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.193@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
I20260805 23:10:53.034794 2569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:53.029188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:37861 (local address 127.25.92.193:46109)
0805 23:10:53.029357 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:53.029374 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:53.029515 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:53.029835 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:53.029838 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:53.029978 (+ 140us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:53.030057 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:53.030064 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:53.031069 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:53.031071 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:53.031600 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:53.031608 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:53.032235 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:53.032237 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:53.032295 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:53.032805 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:10:53.032817 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:53.034691 (+ 1874us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.92.254:37861: Server oryx/127.25.92.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":76,"thread_start_us":109,"threads_started":1}
W20260805 23:10:53.034984 2568 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:10:53.029258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.193:46109 (local address 127.25.92.254:37861)
0805 23:10:53.029369 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:53.029373 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:53.029447 (+ 74us) server_negotiation.cc:407] Connection header received
0805 23:10:53.029645 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:53.029648 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:53.029705 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:53.029783 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:53.030171 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:53.030943 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:53.031714 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:53.032120 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:53.032243 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:53.034855 (+ 2612us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.92.193:46109: BlockingRecv error: recv got EOF from 127.25.92.193:46109 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260805 23:10:53.035066 2566 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260805 23:10:53.131714 2570 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:10:53.131999 2570 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:10:53.132045 2570 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:10:53.136255 2570 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:10:53.136325 2570 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:10:53.136386 2570 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.92.194
I20260805 23:10:53.141912 2570 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:40701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/krb5kdc/oryx_127.25.92.194.keytab
--principal=oryx/127.25.92.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.92.194:39391
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.92.194
--webserver_port=41533
--tserver_master_addrs=127.25.92.254:37861
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2570
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.92.194
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:10:53.143036 2570 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:10:53.144096 2570 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:10:53.154378 2576 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:53.154378 2577 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:53.154374 2579 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:10:53.154845 2570 server_base.cc:1034] running on GCE node
I20260805 23:10:53.155333 2570 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:10:53.155951 2570 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:10:53.157119 2570 hybrid_clock.cc:648] HybridClock initialized: now 1785971453157106 us; error 29 us; skew 500 ppm
Aug 05 23:10:53 dist-test-slave-ss4n krb5kdc[1806](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971453, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.92.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:10:53.160130 2570 init.cc:377] Logged in from keytab as oryx/127.25.92.194@KRBTEST.COM (short username oryx)
I20260805 23:10:53.161105 2570 webserver.cc:460] Webserver started at http://127.25.92.194:41533/ using document root <none> and password file <none>
I20260805 23:10:53.161707 2570 fs_manager.cc:359] Metadata directory not provided
I20260805 23:10:53.161784 2570 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:10:53.165906 2570 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.002s
I20260805 23:10:53.168349 2587 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:53.169624 2570 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:10:53.169754 2570 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/ts-1/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/ts-1/wal
uuid: "42e2c18a010b42aea0b5d893b5191e9d"
format_stamp: "Formatted at 2026-08-05 23:10:52 on dist-test-slave-ss4n"
server_key: "0ba22d9aa7a3d19b4c30f60e2a042398"
server_key_iv: "a1aaa446451f563188bd0bf960724d43"
server_key_version: "encryptionkey@0"
I20260805 23:10:53.170305 2570 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:10:53.186072 2570 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:10:53.189092 2570 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:10:53.189291 2570 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:10:53.189929 2570 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:10:53.190743 2570 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:10:53.190812 2570 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:53.190874 2570 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:10:53.190915 2570 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:53.200286 2570 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.194:39391
I20260805 23:10:53.200309 2703 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.194:39391 every 8 connection(s)
I20260805 23:10:53.201272 2570 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/ts-1/data/info.pb
I20260805 23:10:53.201776 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 2570
I20260805 23:10:53.202236 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.92.195:40521
--local_ip_for_outbound_sockets=127.25.92.195
--tserver_master_addrs=127.25.92.254:37861
--webserver_port=35087
--webserver_interface=127.25.92.195
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.195.keytab
--principal=kudu/127.25.92.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:40701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.25.92.195
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/krb5kdc/oryx_127.25.92.195.keytab with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 05 23:10:53 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.194@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:10:53 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.194@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
W20260805 23:10:53.213928 2568 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:10:53.203437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.194:42897 (local address 127.25.92.254:37861)
0805 23:10:53.203672 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:53.203679 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:53.204546 (+ 867us) server_negotiation.cc:407] Connection header received
0805 23:10:53.205976 (+ 1430us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:53.205982 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:53.206063 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:53.206171 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:53.207324 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:53.208259 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:53.209435 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:53.209937 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:53.210200 (+ 263us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:53.213747 (+ 3547us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.92.194:42897: BlockingRecv error: recv got EOF from 127.25.92.194:42897 (error 108)
Metrics: {"server-negotiator.queue_time_us":131}
I20260805 23:10:53.214461 2707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:53.203902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:37861 (local address 127.25.92.194:42897)
0805 23:10:53.204367 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:53.204419 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:53.205761 (+ 1342us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:53.206316 (+ 555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:53.206323 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:53.206715 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:53.207172 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:53.207184 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:53.208405 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:53.208408 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:53.209306 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:53.209316 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:53.210213 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:53.210216 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:53.210364 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:53.211095 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:10:53.211112 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:53.213569 (+ 2457us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.92.254:37861: Server oryx/127.25.92.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":147,"threads_started":1}
W20260805 23:10:53.215098 2704 heartbeater.cc:644] Failed to heartbeat to 127.25.92.254:37861 (0 consecutive failures): Not authorized: Failed to ping master at 127.25.92.254:37861: Client connection negotiation failed: client connection to 127.25.92.254:37861: Server oryx/127.25.92.254@KRBTEST.COM not found in Kerberos database
Aug 05 23:10:53 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.194@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:10:53 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.194@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
I20260805 23:10:53.221220 2707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:53.215610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:37861 (local address 127.25.92.194:55453)
0805 23:10:53.215725 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:53.215741 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:53.215821 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:53.216091 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:53.216097 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:53.216293 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:53.216393 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:53.216398 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:53.217462 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:53.217465 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:53.218067 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:53.218077 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:53.218682 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:53.218685 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:53.218760 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:53.219283 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:10:53.219295 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:53.221106 (+ 1811us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.92.254:37861: Server oryx/127.25.92.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":46}
W20260805 23:10:53.221398 2568 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:10:53.215652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.194:55453 (local address 127.25.92.254:37861)
0805 23:10:53.215772 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:53.215776 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:53.215791 (+ 15us) server_negotiation.cc:407] Connection header received
0805 23:10:53.215913 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:53.215916 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:53.215961 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:53.216048 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:53.216504 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:53.217348 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:53.218180 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:53.218535 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:53.218639 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:53.221275 (+ 2636us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.92.194:55453: BlockingRecv error: recv got EOF from 127.25.92.194:55453 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 05 23:10:53 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.194@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:10:53 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.194@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
I20260805 23:10:53.227135 2707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:53.221857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:37861 (local address 127.25.92.194:38627)
0805 23:10:53.221953 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:53.221969 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:53.222044 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:53.222295 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:53.222297 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:53.222420 (+ 123us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:53.222499 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:53.222554 (+ 55us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:53.223595 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:53.223597 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:53.224112 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:53.224120 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:53.224682 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:53.224685 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:53.224741 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:53.225255 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:10:53.225266 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:53.227037 (+ 1771us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.92.254:37861: Server oryx/127.25.92.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":28}
W20260805 23:10:53.227308 2568 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:10:53.221930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.194:38627 (local address 127.25.92.254:37861)
0805 23:10:53.222032 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:53.222036 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:53.222049 (+ 13us) server_negotiation.cc:407] Connection header received
0805 23:10:53.222129 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:53.222132 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:53.222185 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:53.222268 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:53.222658 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:53.223456 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:53.224220 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:53.224576 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:53.224680 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:53.227197 (+ 2517us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.92.194:38627: BlockingRecv error: recv got EOF from 127.25.92.194:38627 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260805 23:10:53.227643 2704 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260805 23:10:53.322974 2706 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:10:53.323289 2706 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:10:53.323335 2706 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:10:53.327636 2706 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:10:53.327718 2706 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:10:53.327782 2706 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.92.195
I20260805 23:10:53.333343 2706 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:40701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/krb5kdc/oryx_127.25.92.195.keytab
--principal=oryx/127.25.92.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.92.195:40521
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.92.195
--webserver_port=35087
--tserver_master_addrs=127.25.92.254:37861
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2706
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.92.195
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:10:53.334625 2706 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:10:53.335726 2706 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:10:53.347399 2713 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:53.347419 2716 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:10:53.347399 2714 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:10:53.348218 2706 server_base.cc:1034] running on GCE node
I20260805 23:10:53.348659 2706 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:10:53.349359 2706 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:10:53.350719 2706 hybrid_clock.cc:648] HybridClock initialized: now 1785971453350696 us; error 38 us; skew 500 ppm
Aug 05 23:10:53 dist-test-slave-ss4n krb5kdc[1806](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971453, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.92.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:10:53.353850 2706 init.cc:377] Logged in from keytab as oryx/127.25.92.195@KRBTEST.COM (short username oryx)
I20260805 23:10:53.354925 2706 webserver.cc:460] Webserver started at http://127.25.92.195:35087/ using document root <none> and password file <none>
I20260805 23:10:53.355577 2706 fs_manager.cc:359] Metadata directory not provided
I20260805 23:10:53.355703 2706 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:10:53.359954 2706 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260805 23:10:53.362429 2723 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:53.364017 2706 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:10:53.364200 2706 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/ts-2/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/ts-2/wal
uuid: "7004e2e62537442aac2e12a8ab98d501"
format_stamp: "Formatted at 2026-08-05 23:10:52 on dist-test-slave-ss4n"
server_key: "28b9036997c580e2ce2e7084ac8880cb"
server_key_iv: "856e63828b557c86f6b7aaf92f704e37"
server_key_version: "encryptionkey@0"
I20260805 23:10:53.364852 2706 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:10:53.385560 2706 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:10:53.389205 2706 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:10:53.389449 2706 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:10:53.390213 2706 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:10:53.391122 2706 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:10:53.391204 2706 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:53.391280 2706 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:10:53.391363 2706 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:10:53.402217 2706 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.195:40521
I20260805 23:10:53.402247 2841 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.195:40521 every 8 connection(s)
I20260805 23:10:53.403362 2706 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971406251336-25971-0/minicluster-data/ts-2/data/info.pb
I20260805 23:10:53.410954 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 2706
Aug 05 23:10:53 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.195@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:10:53 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.195@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
W20260805 23:10:53.415850 2568 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:10:53.405275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.195:45271 (local address 127.25.92.254:37861)
0805 23:10:53.405419 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:53.405423 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:53.406425 (+ 1002us) server_negotiation.cc:407] Connection header received
0805 23:10:53.407874 (+ 1449us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:53.407878 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:53.407944 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:53.408047 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:53.409277 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:53.409887 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:53.411178 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:53.411499 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:53.411600 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:53.415744 (+ 4144us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.92.195:45271: BlockingRecv error: recv got EOF from 127.25.92.195:45271 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
I20260805 23:10:53.416476 2844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:53.405719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:37861 (local address 127.25.92.195:45271)
0805 23:10:53.406242 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:53.406287 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:53.407626 (+ 1339us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:53.408230 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:53.408237 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:53.408621 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:53.409110 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:53.409124 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:53.410020 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:53.410025 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:53.410975 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:53.410985 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:53.411699 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:53.411706 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:53.411964 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:53.412828 (+ 864us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:10:53.412845 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:53.415581 (+ 2736us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.92.254:37861: Server oryx/127.25.92.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":128,"threads_started":1}
W20260805 23:10:53.417146 2842 heartbeater.cc:644] Failed to heartbeat to 127.25.92.254:37861 (0 consecutive failures): Not authorized: Failed to ping master at 127.25.92.254:37861: Client connection negotiation failed: client connection to 127.25.92.254:37861: Server oryx/127.25.92.254@KRBTEST.COM not found in Kerberos database
Aug 05 23:10:53 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.195@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:10:53 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.195@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
I20260805 23:10:53.423822 2844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:53.417722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:37861 (local address 127.25.92.195:57651)
0805 23:10:53.417939 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:53.417963 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:53.418035 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:53.418313 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:53.418316 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:53.418515 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:53.418659 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:53.418668 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:53.419434 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:53.419439 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:53.420207 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:53.420218 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:53.420869 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:53.420872 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:53.420968 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:53.421437 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:10:53.421455 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:53.423716 (+ 2261us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.92.254:37861: Server oryx/127.25.92.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":141}
I20260805 23:10:53.423887 2846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:53.412232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49264 (local address 127.25.92.254:37861)
0805 23:10:53.412562 (+ 330us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:53.412566 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:53.412578 (+ 12us) server_negotiation.cc:407] Connection header received
0805 23:10:53.412613 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:53.412615 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:53.412655 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:53.412773 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:53.413707 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:53.414317 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:53.416891 (+ 2574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:53.417258 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:53.417361 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:53.419378 (+ 2017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:10:53.419388 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:10:53.419415 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:10:53.419458 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:10:53.421383 (+ 1925us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:53.422001 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:53.422006 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:53.422011 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:53.422146 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:10:53.422500 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:10:53.422502 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:10:53.422503 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:10:53.422906 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:10:53.423142 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0805 23:10:53.423424 (+ 282us) server_negotiation.cc:299] Negotiation successful
0805 23:10:53.423694 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":246,"thread_start_us":129,"threads_started":1}
W20260805 23:10:53.423916 2568 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:10:53.417845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.195:57651 (local address 127.25.92.254:37861)
0805 23:10:53.418002 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:53.418007 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:53.418021 (+ 14us) server_negotiation.cc:407] Connection header received
0805 23:10:53.418129 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:53.418134 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:53.418184 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:53.418269 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:53.418798 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:53.419320 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:53.420335 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:53.420737 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:53.420850 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:53.423841 (+ 2991us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.92.195:57651: BlockingRecv error: recv got EOF from 127.25.92.195:57651 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 05 23:10:53 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.195@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:10:53 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.195@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
I20260805 23:10:53.430101 2844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:53.424590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:37861 (local address 127.25.92.195:50321)
0805 23:10:53.424715 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:53.424730 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:53.424807 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:53.425080 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:53.425081 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:53.425211 (+ 130us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:53.425281 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:53.425286 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:53.426264 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:53.426267 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:53.426981 (+ 714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:53.426991 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:53.427460 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:53.427465 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:53.427553 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:53.428078 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:10:53.428090 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:53.429977 (+ 1887us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.92.254:37861: Server oryx/127.25.92.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":53}
W20260805 23:10:53.430289 2568 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:10:53.424699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.195:50321 (local address 127.25.92.254:37861)
0805 23:10:53.424836 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:53.424839 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:53.424851 (+ 12us) server_negotiation.cc:407] Connection header received
0805 23:10:53.424903 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:53.424906 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:53.424962 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:53.425056 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:53.425392 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:53.426138 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:53.427102 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:53.427338 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:53.427455 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:53.430123 (+ 2668us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.92.195:50321: BlockingRecv error: recv got EOF from 127.25.92.195:50321 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260805 23:10:53.430370 2842 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 05 23:10:54 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.193@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:10:54 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.193@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
I20260805 23:10:54.042224 2847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:54.035727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:37861 (local address 127.25.92.193:42033)
0805 23:10:54.036014 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:54.036031 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:54.036125 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:54.036464 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:54.036467 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:54.036713 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:54.036830 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:54.036837 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:54.037772 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:54.037775 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:54.038433 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:54.038443 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:54.039034 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:54.039038 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:54.039133 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:54.039734 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:10:54.039755 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:54.042007 (+ 2252us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.92.254:37861: Server oryx/127.25.92.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":90,"threads_started":1}
W20260805 23:10:54.042429 2848 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:10:54.035821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.193:42033 (local address 127.25.92.254:37861)
0805 23:10:54.036105 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:54.036108 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:54.036121 (+ 13us) server_negotiation.cc:407] Connection header received
0805 23:10:54.036261 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:54.036264 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:54.036322 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:54.036406 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:54.036954 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:54.037657 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:54.038605 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:54.038879 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:54.038980 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:54.042272 (+ 3292us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.92.193:42033: BlockingRecv error: recv got EOF from 127.25.92.193:42033 (error 108)
Metrics: {"server-negotiator.queue_time_us":219,"thread_start_us":94,"threads_started":1}
Aug 05 23:10:54 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.194@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:10:54 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.194@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
I20260805 23:10:54.234777 2849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:54.228545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:37861 (local address 127.25.92.194:37413)
0805 23:10:54.228908 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:54.228929 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:54.229098 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:54.229443 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:54.229445 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:54.229761 (+ 316us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:54.229891 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:54.229900 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:54.230737 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:54.230740 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:54.231403 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:54.231411 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:54.231890 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:54.231892 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:54.231958 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:54.232489 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:10:54.232501 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:54.234645 (+ 2144us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.92.254:37861: Server oryx/127.25.92.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":123,"threads_started":1}
W20260805 23:10:54.234992 2848 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:10:54.228724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.194:37413 (local address 127.25.92.254:37861)
0805 23:10:54.228899 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:54.228903 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:54.229026 (+ 123us) server_negotiation.cc:407] Connection header received
0805 23:10:54.229226 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:54.229229 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:54.229281 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:54.229404 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:54.230026 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:54.230621 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:54.231543 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:54.231778 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:54.231869 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:54.234812 (+ 2943us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.92.194:37413: BlockingRecv error: recv got EOF from 127.25.92.194:37413 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Aug 05 23:10:54 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.195@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:10:54 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.195@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
I20260805 23:10:54.437459 2850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:54.431158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:37861 (local address 127.25.92.195:58767)
0805 23:10:54.431511 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:54.431527 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:54.431642 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:54.431884 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:54.431886 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:54.432069 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:54.432163 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:54.432169 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:54.433168 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:54.433171 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:54.433886 (+ 715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:54.433898 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:54.434402 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:54.434405 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:54.434532 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:54.435170 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:10:54.435185 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:54.437332 (+ 2147us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.92.254:37861: Server oryx/127.25.92.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":103,"threads_started":1}
W20260805 23:10:54.437672 2848 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:10:54.431286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.195:58767 (local address 127.25.92.254:37861)
0805 23:10:54.431518 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:54.431530 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:54.431664 (+ 134us) server_negotiation.cc:407] Connection header received
0805 23:10:54.431716 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:54.431718 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:54.431769 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:54.431852 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:54.432285 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:54.432872 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:54.434027 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:54.434281 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:54.434379 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:54.437541 (+ 3162us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.92.195:58767: BlockingRecv error: recv got EOF from 127.25.92.195:58767 (error 108)
Metrics: {"server-negotiator.queue_time_us":119}
Aug 05 23:10:55 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.193@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:10:55 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.193@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
I20260805 23:10:55.055245 2851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:55.043244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:37861 (local address 127.25.92.193:58653)
0805 23:10:55.043540 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:55.043555 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:55.043666 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:55.044035 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:55.044038 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:55.044235 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:55.044343 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:55.044349 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:55.045407 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:55.045411 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:55.046245 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:55.046264 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:55.046985 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:55.046988 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:55.047059 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:55.052575 (+ 5516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:10:55.052592 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:55.055068 (+ 2476us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.92.254:37861: Server oryx/127.25.92.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":100,"threads_started":1}
W20260805 23:10:55.055401 2852 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:10:55.043385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.193:58653 (local address 127.25.92.254:37861)
0805 23:10:55.043742 (+ 357us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:55.043746 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:55.043763 (+ 17us) server_negotiation.cc:407] Connection header received
0805 23:10:55.043811 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:55.043814 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:55.043861 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:55.043964 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:55.044531 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:55.045286 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:55.046445 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:55.046845 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:55.046973 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:55.055262 (+ 8289us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.92.193:58653: BlockingRecv error: recv got EOF from 127.25.92.193:58653 (error 108)
Metrics: {"server-negotiator.queue_time_us":213,"thread_start_us":118,"threads_started":1}
Aug 05 23:10:55 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.194@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:10:55 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.194@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
I20260805 23:10:55.242038 2853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:55.235587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:37861 (local address 127.25.92.194:36733)
0805 23:10:55.235873 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:55.235887 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:55.235974 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:55.236429 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:55.236431 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:55.236659 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:55.236760 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:55.236766 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:55.237791 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:55.237794 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:55.238429 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:55.238438 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:55.238948 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:55.238952 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:55.239021 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:55.239566 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:10:55.239578 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:55.241897 (+ 2319us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.92.254:37861: Server oryx/127.25.92.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":110,"threads_started":1}
W20260805 23:10:55.242292 2852 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:10:55.235732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.194:36733 (local address 127.25.92.254:37861)
0805 23:10:55.235906 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:55.235910 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:55.235928 (+ 18us) server_negotiation.cc:407] Connection header received
0805 23:10:55.236170 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:55.236173 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:55.236250 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:55.236343 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:55.236904 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:55.237531 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:55.238592 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:55.238837 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:55.238928 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:55.242072 (+ 3144us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.92.194:36733: BlockingRecv error: recv got EOF from 127.25.92.194:36733 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 05 23:10:55 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.195@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:10:55 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.195@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
I20260805 23:10:55.450574 2854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:55.438411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:37861 (local address 127.25.92.195:60305)
0805 23:10:55.438783 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:55.438797 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:55.438906 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:55.439198 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:55.439200 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:55.439378 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:55.439478 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:55.439484 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:55.440440 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:55.440444 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:55.441112 (+ 668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:55.441122 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:55.441853 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:55.441858 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:55.441961 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:55.448015 (+ 6054us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:10:55.448032 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:55.450361 (+ 2329us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.92.254:37861: Server oryx/127.25.92.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":183,"threads_started":1}
W20260805 23:10:55.450784 2852 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:10:55.438659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.195:60305 (local address 127.25.92.254:37861)
0805 23:10:55.438798 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:55.438801 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:55.438867 (+ 66us) server_negotiation.cc:407] Connection header received
0805 23:10:55.438993 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:55.439019 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:55.439070 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:55.439152 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:55.439637 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:55.440302 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:55.441310 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:55.441685 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:55.441849 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:55.450604 (+ 8755us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.92.195:60305: BlockingRecv error: recv got EOF from 127.25.92.195:60305 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 05 23:10:56 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.193@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:10:56 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.193@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
I20260805 23:10:56.063498 2855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:56.056434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:37861 (local address 127.25.92.193:60877)
0805 23:10:56.056943 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:56.056960 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:56.057066 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:56.057442 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:56.057445 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:56.057725 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:56.057827 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:56.057834 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:56.058787 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:56.058790 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:56.059507 (+ 717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:56.059519 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:56.060091 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:56.060098 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:56.060205 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:56.060997 (+ 792us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:10:56.061015 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:56.063320 (+ 2305us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.92.254:37861: Server oryx/127.25.92.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":269,"threads_started":1}
W20260805 23:10:56.063620 2856 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:10:56.056693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.193:60877 (local address 127.25.92.254:37861)
0805 23:10:56.056991 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:56.057002 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:56.057019 (+ 17us) server_negotiation.cc:407] Connection header received
0805 23:10:56.057216 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:56.057219 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:56.057272 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:56.057366 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:56.057972 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:56.058637 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:56.059671 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:56.059953 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:56.060049 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:56.063462 (+ 3413us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.92.193:60877: BlockingRecv error: recv got EOF from 127.25.92.193:60877 (error 108)
Metrics: {"server-negotiator.queue_time_us":222,"thread_start_us":66,"threads_started":1}
Aug 05 23:10:56 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.194@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:10:56 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.194@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
I20260805 23:10:56.249454 2857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:56.243232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:37861 (local address 127.25.92.194:37473)
0805 23:10:56.243557 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:56.243575 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:56.243734 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:56.244027 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:56.244031 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:56.244273 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:56.244370 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:56.244376 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:56.245207 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:56.245213 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:56.245914 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:56.245924 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:56.246412 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:56.246415 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:56.246505 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:56.247102 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:10:56.247114 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:56.249244 (+ 2130us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.92.254:37861: Server oryx/127.25.92.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":117,"threads_started":1}
W20260805 23:10:56.249527 2856 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:10:56.243303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.194:37473 (local address 127.25.92.254:37861)
0805 23:10:56.243515 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:56.243520 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:56.243680 (+ 160us) server_negotiation.cc:407] Connection header received
0805 23:10:56.243852 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:56.243855 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:56.243914 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:56.244003 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:56.244500 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:56.245085 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:56.246059 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:56.246290 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:56.246397 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:56.249392 (+ 2995us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.92.194:37473: BlockingRecv error: recv got EOF from 127.25.92.194:37473 (error 108)
Metrics: {"server-negotiator.queue_time_us":114}
Aug 05 23:10:56 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.195@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:10:56 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.195@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
I20260805 23:10:56.457685 2858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:56.451506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:37861 (local address 127.25.92.195:50545)
0805 23:10:56.451844 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:56.451862 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:56.451990 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:56.452297 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:56.452301 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:56.452587 (+ 286us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:56.452747 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:56.452758 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:56.453721 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:56.453726 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:56.454306 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:56.454314 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:56.454831 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:56.454834 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:56.454914 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:56.455521 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:10:56.455544 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:56.457465 (+ 1921us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.92.254:37861: Server oryx/127.25.92.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":111,"threads_started":1}
W20260805 23:10:56.457899 2856 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:10:56.451678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.195:50545 (local address 127.25.92.254:37861)
0805 23:10:56.451831 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:56.451835 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:56.451978 (+ 143us) server_negotiation.cc:407] Connection header received
0805 23:10:56.452095 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:56.452101 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:56.452173 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:56.452241 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:56.452901 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:56.453611 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:56.454435 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:56.454674 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:56.454795 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:56.457716 (+ 2921us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.92.195:50545: BlockingRecv error: recv got EOF from 127.25.92.195:50545 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 05 23:10:57 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.193@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:10:57 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.193@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
I20260805 23:10:57.070442 2859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:57.064467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:37861 (local address 127.25.92.193:57205)
0805 23:10:57.064770 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:57.064791 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:57.064886 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:57.065171 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:57.065174 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:57.065355 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:57.065453 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:57.065459 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:57.066349 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:57.066352 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:57.066931 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:57.066941 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:57.067466 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:57.067469 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:57.067544 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:57.068142 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:10:57.068155 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:57.070242 (+ 2087us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.92.254:37861: Server oryx/127.25.92.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":121,"threads_started":1}
W20260805 23:10:57.070591 2860 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:10:57.064669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.193:57205 (local address 127.25.92.254:37861)
0805 23:10:57.064923 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:57.064927 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:57.064939 (+ 12us) server_negotiation.cc:407] Connection header received
0805 23:10:57.064981 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:57.064983 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:57.065028 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:57.065116 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:57.065634 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:57.066228 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:57.067107 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:57.067344 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:57.067438 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:57.070378 (+ 2940us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.92.193:57205: BlockingRecv error: recv got EOF from 127.25.92.193:57205 (error 108)
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":88,"threads_started":1}
Aug 05 23:10:57 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.194@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:10:57 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.194@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
I20260805 23:10:57.256466 2861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:57.250471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:37861 (local address 127.25.92.194:40773)
0805 23:10:57.250849 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:57.250866 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:57.250983 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:57.251240 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:57.251242 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:57.251418 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:57.251513 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:57.251518 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:57.252295 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:57.252297 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:57.252859 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:57.252866 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:57.253309 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:57.253311 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:57.253372 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:57.254040 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:10:57.254052 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:57.256281 (+ 2229us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.92.254:37861: Server oryx/127.25.92.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":97,"threads_started":1}
W20260805 23:10:57.256621 2860 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:10:57.250662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.194:40773 (local address 127.25.92.254:37861)
0805 23:10:57.250827 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:57.250835 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:57.250951 (+ 116us) server_negotiation.cc:407] Connection header received
0805 23:10:57.251078 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:57.251081 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:57.251130 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:57.251202 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:57.251614 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:57.252194 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:57.252967 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:57.253210 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:57.253296 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:57.256424 (+ 3128us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.92.194:40773: BlockingRecv error: recv got EOF from 127.25.92.194:40773 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 05 23:10:57 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.195@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:10:57 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.195@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
I20260805 23:10:57.464704 2862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:57.458437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:37861 (local address 127.25.92.195:50715)
0805 23:10:57.458807 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:57.458822 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:57.458978 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:57.459285 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:57.459288 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:57.459498 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:57.459591 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:57.459596 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:57.460403 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:57.460410 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:57.460976 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:57.460984 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:57.461471 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:57.461474 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:57.461595 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:57.462200 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:10:57.462212 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:57.464581 (+ 2369us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.92.254:37861: Server oryx/127.25.92.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":89,"threads_started":1}
W20260805 23:10:57.464898 2860 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:10:57.458567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.195:50715 (local address 127.25.92.254:37861)
0805 23:10:57.458773 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:57.458778 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:57.458911 (+ 133us) server_negotiation.cc:407] Connection header received
0805 23:10:57.459081 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:57.459085 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:57.459134 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:57.459259 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:57.459744 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:57.460289 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:57.461101 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:57.461351 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:57.461467 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:57.464752 (+ 3285us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.92.195:50715: BlockingRecv error: recv got EOF from 127.25.92.195:50715 (error 108)
Metrics: {"server-negotiator.queue_time_us":95}
Aug 05 23:10:58 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.193@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:10:58 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.193@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
I20260805 23:10:58.077920 2863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:58.071488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:37861 (local address 127.25.92.193:59077)
0805 23:10:58.071770 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:58.071786 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:58.071886 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:58.072236 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:58.072239 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:58.072503 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:58.072685 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:58.072692 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:58.073668 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:58.073671 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:58.074285 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:58.074338 (+ 53us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:58.074818 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:58.074821 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:58.074884 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:58.075434 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:10:58.075446 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:58.077741 (+ 2295us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.92.254:37861: Server oryx/127.25.92.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":97,"threads_started":1}
W20260805 23:10:58.078027 2864 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:10:58.071594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.193:59077 (local address 127.25.92.254:37861)
0805 23:10:58.071880 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:58.071889 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:58.071912 (+ 23us) server_negotiation.cc:407] Connection header received
0805 23:10:58.071988 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:58.071991 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:58.072052 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:58.072176 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:58.072846 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:58.073534 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:58.074466 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:58.074706 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:58.074795 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:58.077879 (+ 3084us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.92.193:59077: BlockingRecv error: recv got EOF from 127.25.92.193:59077 (error 108)
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":112,"threads_started":1}
Aug 05 23:10:58 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.194@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:10:58 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.194@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
I20260805 23:10:58.265174 2865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:58.257822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:37861 (local address 127.25.92.194:36995)
0805 23:10:58.258141 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:58.258155 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:58.258255 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:58.258521 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:58.258524 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:58.258733 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:58.258841 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:58.258847 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:58.259808 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:58.259814 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:58.260791 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:58.260818 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:58.261447 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:58.261453 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:58.261600 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:58.262704 (+ 1104us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:10:58.262722 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:58.265048 (+ 2326us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.92.254:37861: Server oryx/127.25.92.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":121,"threads_started":1}
W20260805 23:10:58.265431 2864 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:10:58.257968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.194:36995 (local address 127.25.92.254:37861)
0805 23:10:58.258154 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:58.258157 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:58.258207 (+ 50us) server_negotiation.cc:407] Connection header received
0805 23:10:58.258347 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:58.258350 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:58.258398 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:58.258480 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:58.258952 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:58.259639 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:58.260962 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:58.261304 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:58.261438 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:58.265271 (+ 3833us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.92.194:36995: BlockingRecv error: recv got EOF from 127.25.92.194:36995 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 05 23:10:58 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.195@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:10:58 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.195@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
I20260805 23:10:58.472586 2866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:58.465841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:37861 (local address 127.25.92.195:38541)
0805 23:10:58.466184 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:58.466203 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:58.466334 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:58.466683 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:58.466686 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:58.466894 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:58.467003 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:58.467009 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:58.467821 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:58.467823 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:58.468420 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:58.468429 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:58.469174 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:58.469178 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:58.469300 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:58.470073 (+ 773us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:10:58.470089 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:58.472446 (+ 2357us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.92.254:37861: Server oryx/127.25.92.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":185,"threads_started":1}
W20260805 23:10:58.472839 2864 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:10:58.465999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.195:38541 (local address 127.25.92.254:37861)
0805 23:10:58.466145 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:58.466148 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:58.466302 (+ 154us) server_negotiation.cc:407] Connection header received
0805 23:10:58.466427 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:58.466429 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:58.466482 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:58.466568 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:58.467151 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:58.467703 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:58.468612 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:58.468996 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:58.469112 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:58.472709 (+ 3597us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.92.195:38541: BlockingRecv error: recv got EOF from 127.25.92.195:38541 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 05 23:10:59 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.193@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:10:59 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.193@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
I20260805 23:10:59.085278 2867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:59.078858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:37861 (local address 127.25.92.193:34981)
0805 23:10:59.079258 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:59.079278 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:59.079426 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:59.079816 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:59.079821 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:59.080087 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:59.080224 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:59.080233 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:59.081214 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:59.081217 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:59.081922 (+ 705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:59.081932 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:59.082443 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:59.082453 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:59.082535 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:59.083065 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:10:59.083079 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:59.085158 (+ 2079us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.92.254:37861: Server oryx/127.25.92.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":90,"threads_started":1}
W20260805 23:10:59.085513 2868 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:10:59.079007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.193:34981 (local address 127.25.92.254:37861)
0805 23:10:59.079265 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:59.079268 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:59.079390 (+ 122us) server_negotiation.cc:407] Connection header received
0805 23:10:59.079608 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:59.079617 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:59.079673 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:59.079751 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:59.080377 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:59.081099 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:59.082051 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:59.082316 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:59.082397 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:59.085338 (+ 2941us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.92.193:34981: BlockingRecv error: recv got EOF from 127.25.92.193:34981 (error 108)
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":62,"threads_started":1}
Aug 05 23:10:59 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.194@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:10:59 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.194@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
I20260805 23:10:59.272544 2869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:59.266153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:37861 (local address 127.25.92.194:39481)
0805 23:10:59.266478 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:59.266494 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:59.266618 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:59.267317 (+ 699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:59.267321 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:59.267538 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:59.267632 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:59.267638 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:59.268437 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:59.268443 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:59.269093 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:59.269103 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:59.269633 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:59.269636 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:59.269718 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:59.270274 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:10:59.270286 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:59.272376 (+ 2090us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.92.254:37861: Server oryx/127.25.92.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":90,"threads_started":1}
W20260805 23:10:59.272722 2868 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:10:59.266868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.194:39481 (local address 127.25.92.254:37861)
0805 23:10:59.267036 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:59.267042 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:59.267057 (+ 15us) server_negotiation.cc:407] Connection header received
0805 23:10:59.267102 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:59.267104 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:59.267156 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:59.267251 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:59.267765 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:59.268318 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:59.269218 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:59.269465 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:59.269633 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:59.272570 (+ 2937us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.92.194:39481: BlockingRecv error: recv got EOF from 127.25.92.194:39481 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 05 23:10:59 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.195@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:10:59 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.195@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
I20260805 23:10:59.479748 2870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:10:59.473526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:37861 (local address 127.25.92.195:54639)
0805 23:10:59.474002 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0805 23:10:59.474021 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0805 23:10:59.474148 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:10:59.474398 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:10:59.474400 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:10:59.474647 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:10:59.474752 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:59.474759 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:59.475505 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:59.475509 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:59.476150 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:10:59.476159 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:10:59.476624 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:10:59.476627 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:10:59.476697 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:59.477264 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:10:59.477276 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:10:59.479598 (+ 2322us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.92.254:37861: Server oryx/127.25.92.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":132,"threads_started":1}
W20260805 23:10:59.480033 2868 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:10:59.473775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.195:54639 (local address 127.25.92.254:37861)
0805 23:10:59.473919 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0805 23:10:59.473923 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:10:59.474112 (+ 189us) server_negotiation.cc:407] Connection header received
0805 23:10:59.474237 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:10:59.474239 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:10:59.474287 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:10:59.474358 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:10:59.474862 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:59.475386 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:59.476257 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:10:59.476519 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:10:59.476599 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:10:59.479890 (+ 3291us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.92.195:54639: BlockingRecv error: recv got EOF from 127.25.92.195:54639 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 05 23:11:00 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.193@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:11:00 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.193@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
I20260805 23:11:00.093024 2871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:11:00.086278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:37861 (local address 127.25.92.193:42259)
0805 23:11:00.086725 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0805 23:11:00.086742 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0805 23:11:00.086868 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:11:00.087192 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:11:00.087195 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:11:00.087397 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:11:00.087584 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:11:00.087592 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:11:00.088440 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:11:00.088443 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:11:00.089081 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:11:00.089092 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:11:00.089779 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:11:00.089782 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:11:00.089859 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:11:00.090518 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:11:00.090533 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:11:00.092898 (+ 2365us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.92.254:37861: Server oryx/127.25.92.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":96,"threads_started":1}
W20260805 23:11:00.093269 2872 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:11:00.086425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.193:42259 (local address 127.25.92.254:37861)
0805 23:11:00.086756 (+ 331us) server_negotiation.cc:206] Beginning negotiation
0805 23:11:00.086760 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:11:00.086807 (+ 47us) server_negotiation.cc:407] Connection header received
0805 23:11:00.086981 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:11:00.086983 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:11:00.087030 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:11:00.087117 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:11:00.087729 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:11:00.088293 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:11:00.089233 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:11:00.089595 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:11:00.089716 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:11:00.093097 (+ 3381us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.92.193:42259: BlockingRecv error: recv got EOF from 127.25.92.193:42259 (error 108)
Metrics: {"server-negotiator.queue_time_us":255,"thread_start_us":114,"threads_started":1}
Aug 05 23:11:00 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.194@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:11:00 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.194@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
I20260805 23:11:00.279750 2873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:11:00.273780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:37861 (local address 127.25.92.194:43801)
0805 23:11:00.274147 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0805 23:11:00.274164 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0805 23:11:00.274271 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:11:00.274550 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:11:00.274553 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:11:00.274768 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:11:00.274861 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:11:00.274867 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:11:00.275732 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:11:00.275738 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:11:00.276307 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:11:00.276315 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:11:00.276803 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:11:00.276807 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:11:00.276894 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:11:00.277428 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:11:00.277439 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:11:00.279624 (+ 2185us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.92.254:37861: Server oryx/127.25.92.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":126,"threads_started":1}
W20260805 23:11:00.279937 2872 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:11:00.273978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.194:43801 (local address 127.25.92.254:37861)
0805 23:11:00.274265 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0805 23:11:00.274271 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:11:00.274286 (+ 15us) server_negotiation.cc:407] Connection header received
0805 23:11:00.274381 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:11:00.274384 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:11:00.274440 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:11:00.274525 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:11:00.274996 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:11:00.275611 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:11:00.276444 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:11:00.276688 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:11:00.276798 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:11:00.279773 (+ 2975us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.92.194:43801: BlockingRecv error: recv got EOF from 127.25.92.194:43801 (error 108)
Metrics: {"server-negotiator.queue_time_us":128}
Aug 05 23:11:00 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.195@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:11:00 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.195@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
I20260805 23:11:00.492416 2874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:11:00.480709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:37861 (local address 127.25.92.195:35637)
0805 23:11:00.481022 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0805 23:11:00.481035 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0805 23:11:00.481187 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:11:00.481457 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:11:00.481460 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:11:00.481693 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:11:00.481787 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:11:00.481793 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:11:00.482577 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:11:00.482582 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:11:00.483152 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:11:00.483161 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:11:00.483640 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:11:00.483644 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:11:00.483725 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:11:00.489665 (+ 5940us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:11:00.489678 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:11:00.492279 (+ 2601us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.92.254:37861: Server oryx/127.25.92.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":94,"threads_started":1}
W20260805 23:11:00.492548 2872 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:11:00.480701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.195:35637 (local address 127.25.92.254:37861)
0805 23:11:00.480857 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0805 23:11:00.480861 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:11:00.481117 (+ 256us) server_negotiation.cc:407] Connection header received
0805 23:11:00.481288 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:11:00.481291 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:11:00.481341 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:11:00.481435 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:11:00.481911 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:11:00.482464 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:11:00.483283 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:11:00.483525 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:11:00.483642 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:11:00.492430 (+ 8788us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.92.195:35637: BlockingRecv error: recv got EOF from 127.25.92.195:35637 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 05 23:11:01 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.193@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:11:01 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.193@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
I20260805 23:11:01.100292 2875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:11:01.094030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:37861 (local address 127.25.92.193:60119)
0805 23:11:01.094585 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0805 23:11:01.094603 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0805 23:11:01.094724 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:11:01.095088 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:11:01.095091 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:11:01.095328 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:11:01.095470 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:11:01.095478 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:11:01.096408 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:11:01.096411 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:11:01.097048 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:11:01.097057 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:11:01.097658 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:11:01.097662 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:11:01.097742 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:11:01.098262 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:11:01.098274 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:11:01.100171 (+ 1897us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.92.254:37861: Server oryx/127.25.92.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":126,"threads_started":1}
W20260805 23:11:01.100510 2876 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:11:01.094293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.193:60119 (local address 127.25.92.254:37861)
0805 23:11:01.094624 (+ 331us) server_negotiation.cc:206] Beginning negotiation
0805 23:11:01.094628 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:11:01.094659 (+ 31us) server_negotiation.cc:407] Connection header received
0805 23:11:01.094850 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:11:01.094853 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:11:01.094910 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:11:01.095016 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:11:01.095637 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:11:01.096295 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:11:01.097168 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:11:01.097399 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:11:01.097497 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:11:01.100349 (+ 2852us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.92.193:60119: BlockingRecv error: recv got EOF from 127.25.92.193:60119 (error 108)
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":70,"threads_started":1}
Aug 05 23:11:01 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.194@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:11:01 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.194@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
I20260805 23:11:01.286559 2877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:11:01.280650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:37861 (local address 127.25.92.194:55663)
0805 23:11:01.280973 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0805 23:11:01.280990 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0805 23:11:01.281137 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:11:01.281414 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:11:01.281417 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:11:01.281665 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:11:01.281758 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:11:01.281764 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:11:01.282756 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:11:01.282761 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:11:01.283319 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:11:01.283327 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:11:01.283807 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:11:01.283810 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:11:01.283897 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:11:01.284487 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:11:01.284500 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:11:01.286441 (+ 1941us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.92.254:37861: Server oryx/127.25.92.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":89,"threads_started":1}
W20260805 23:11:01.286767 2876 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:11:01.280767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.194:55663 (local address 127.25.92.254:37861)
0805 23:11:01.280956 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0805 23:11:01.280962 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:11:01.281078 (+ 116us) server_negotiation.cc:407] Connection header received
0805 23:11:01.281247 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:11:01.281250 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:11:01.281306 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:11:01.281394 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:11:01.281889 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:11:01.282638 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:11:01.283446 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:11:01.283688 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:11:01.283801 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:11:01.286589 (+ 2788us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.92.194:55663: BlockingRecv error: recv got EOF from 127.25.92.194:55663 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 05 23:11:01 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.195@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:11:01 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.195@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
W20260805 23:11:01.500401 2876 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:11:01.493622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.195:56429 (local address 127.25.92.254:37861)
0805 23:11:01.493807 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0805 23:11:01.493812 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:11:01.493983 (+ 171us) server_negotiation.cc:407] Connection header received
0805 23:11:01.494176 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:11:01.494179 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:11:01.494243 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:11:01.494340 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:11:01.494849 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:11:01.495456 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:11:01.496529 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:11:01.496831 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:11:01.496949 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:11:01.500287 (+ 3338us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.92.195:56429: BlockingRecv error: recv got EOF from 127.25.92.195:56429 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
I20260805 23:11:01.500185 2878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:11:01.493439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:37861 (local address 127.25.92.195:56429)
0805 23:11:01.493852 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0805 23:11:01.493877 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0805 23:11:01.494028 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:11:01.494365 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:11:01.494368 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:11:01.494609 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:11:01.494708 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:11:01.494714 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:11:01.495594 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:11:01.495599 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:11:01.496393 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:11:01.496405 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:11:01.497029 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:11:01.497035 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:11:01.497144 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:11:01.497852 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:11:01.497887 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:11:01.500059 (+ 2172us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.92.254:37861: Server oryx/127.25.92.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":125,"threads_started":1}
Aug 05 23:11:02 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.193@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:11:02 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.193@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
I20260805 23:11:02.108165 2879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:11:02.101278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:37861 (local address 127.25.92.193:55391)
0805 23:11:02.101720 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0805 23:11:02.101736 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0805 23:11:02.101844 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:11:02.102158 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:11:02.102161 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:11:02.102377 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:11:02.102483 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:11:02.102490 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:11:02.103490 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:11:02.103544 (+ 54us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:11:02.104144 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:11:02.104154 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:11:02.104830 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:11:02.104834 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:11:02.104939 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:11:02.105751 (+ 812us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:11:02.105770 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:11:02.108041 (+ 2271us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.92.254:37861: Server oryx/127.25.92.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":111,"threads_started":1}
W20260805 23:11:02.108399 2880 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:11:02.101445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.193:55391 (local address 127.25.92.254:37861)
0805 23:11:02.101770 (+ 325us) server_negotiation.cc:206] Beginning negotiation
0805 23:11:02.101774 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:11:02.101793 (+ 19us) server_negotiation.cc:407] Connection header received
0805 23:11:02.101929 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:11:02.101931 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:11:02.101979 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:11:02.102069 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:11:02.102699 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:11:02.103314 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:11:02.104341 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:11:02.104709 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:11:02.104821 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:11:02.108229 (+ 3408us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.92.193:55391: BlockingRecv error: recv got EOF from 127.25.92.193:55391 (error 108)
Metrics: {"server-negotiator.queue_time_us":230,"thread_start_us":66,"threads_started":1}
Aug 05 23:11:02 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.194@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:11:02 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.194@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
I20260805 23:11:02.294276 2881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:11:02.287568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:37861 (local address 127.25.92.194:53811)
0805 23:11:02.287889 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0805 23:11:02.287907 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0805 23:11:02.288043 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:11:02.288366 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:11:02.288370 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:11:02.288606 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:11:02.288735 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:11:02.288744 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:11:02.289733 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:11:02.289739 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:11:02.290684 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:11:02.290694 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:11:02.291314 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:11:02.291317 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:11:02.291396 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:11:02.292055 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:11:02.292070 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:11:02.294158 (+ 2088us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.92.254:37861: Server oryx/127.25.92.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":94,"threads_started":1}
W20260805 23:11:02.294417 2880 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:11:02.287670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.194:53811 (local address 127.25.92.254:37861)
0805 23:11:02.287906 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0805 23:11:02.287912 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:11:02.287983 (+ 71us) server_negotiation.cc:407] Connection header received
0805 23:11:02.288171 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:11:02.288178 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:11:02.288240 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:11:02.288335 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:11:02.288859 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:11:02.289641 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:11:02.290808 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:11:02.291192 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:11:02.291310 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:11:02.294298 (+ 2988us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.92.194:53811: BlockingRecv error: recv got EOF from 127.25.92.194:53811 (error 108)
Metrics: {"server-negotiator.queue_time_us":136}
Aug 05 23:11:02 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.195@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:11:02 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.195@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
I20260805 23:11:02.507135 2882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:11:02.501399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:37861 (local address 127.25.92.195:37845)
0805 23:11:02.501732 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0805 23:11:02.501747 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0805 23:11:02.501847 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:11:02.502096 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:11:02.502098 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:11:02.502290 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:11:02.502384 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:11:02.502389 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:11:02.503167 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:11:02.503169 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:11:02.503740 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:11:02.503747 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:11:02.504209 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:11:02.504212 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:11:02.504278 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:11:02.504844 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:11:02.504857 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:11:02.506988 (+ 2131us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.92.254:37861: Server oryx/127.25.92.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":142,"threads_started":1}
W20260805 23:11:02.507275 2880 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:11:02.501550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.195:37845 (local address 127.25.92.254:37861)
0805 23:11:02.501752 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0805 23:11:02.501756 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:11:02.501800 (+ 44us) server_negotiation.cc:407] Connection header received
0805 23:11:02.501933 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:11:02.501935 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:11:02.501982 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:11:02.502052 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:11:02.502484 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:11:02.503062 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:11:02.503866 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:11:02.504097 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:11:02.504185 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:11:02.507140 (+ 2955us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.92.195:37845: BlockingRecv error: recv got EOF from 127.25.92.195:37845 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
Aug 05 23:11:03 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.193@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:11:03 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.193@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
I20260805 23:11:03.114754 2883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:11:03.109059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:37861 (local address 127.25.92.193:47185)
0805 23:11:03.109367 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0805 23:11:03.109381 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0805 23:11:03.109469 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:11:03.109784 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:11:03.109786 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:11:03.109968 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:11:03.110067 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:11:03.110073 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:11:03.110951 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:11:03.110954 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:11:03.111521 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:11:03.111529 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:11:03.111997 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:11:03.111999 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:11:03.112060 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:11:03.112583 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:11:03.112596 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:11:03.114566 (+ 1970us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.92.254:37861: Server oryx/127.25.92.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":107,"threads_started":1}
W20260805 23:11:03.114889 2884 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:11:03.109173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.193:47185 (local address 127.25.92.254:37861)
0805 23:11:03.109441 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0805 23:11:03.109445 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:11:03.109460 (+ 15us) server_negotiation.cc:407] Connection header received
0805 23:11:03.109612 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:11:03.109615 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:11:03.109659 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:11:03.109743 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:11:03.110223 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:11:03.110830 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:11:03.111646 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:11:03.111887 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:11:03.111979 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:11:03.114748 (+ 2769us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.92.193:47185: BlockingRecv error: recv got EOF from 127.25.92.193:47185 (error 108)
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":83,"threads_started":1}
Aug 05 23:11:03 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.194@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:11:03 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.194@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
I20260805 23:11:03.301129 2885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:11:03.295295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:37861 (local address 127.25.92.194:42159)
0805 23:11:03.295639 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0805 23:11:03.295654 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0805 23:11:03.295795 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:11:03.296074 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:11:03.296077 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:11:03.296298 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:11:03.296390 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:11:03.296396 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:11:03.297223 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:11:03.297229 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:11:03.297880 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:11:03.297889 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:11:03.298370 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:11:03.298373 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:11:03.298456 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:11:03.299006 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:11:03.299017 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:11:03.300985 (+ 1968us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.92.254:37861: Server oryx/127.25.92.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":102,"threads_started":1}
W20260805 23:11:03.301330 2884 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:11:03.295386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.194:42159 (local address 127.25.92.254:37861)
0805 23:11:03.295571 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0805 23:11:03.295576 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:11:03.295738 (+ 162us) server_negotiation.cc:407] Connection header received
0805 23:11:03.295907 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:11:03.295910 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:11:03.295967 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:11:03.296055 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:11:03.296519 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:11:03.297085 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:11:03.298005 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:11:03.298253 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:11:03.298368 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:11:03.301160 (+ 2792us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.92.194:42159: BlockingRecv error: recv got EOF from 127.25.92.194:42159 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 05 23:11:03 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.195@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:11:03 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.195@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
I20260805 23:11:03.513768 2886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:11:03.508079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:37861 (local address 127.25.92.195:54671)
0805 23:11:03.508379 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0805 23:11:03.508394 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0805 23:11:03.508530 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:11:03.508838 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:11:03.508840 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:11:03.509024 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:11:03.509118 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:11:03.509123 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:11:03.509947 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:11:03.509950 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:11:03.510503 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:11:03.510511 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:11:03.510958 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:11:03.510961 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:11:03.511025 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:11:03.511609 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:11:03.511621 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:11:03.513604 (+ 1983us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.92.254:37861: Server oryx/127.25.92.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":126,"threads_started":1}
W20260805 23:11:03.514108 2884 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:11:03.508255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.195:54671 (local address 127.25.92.254:37861)
0805 23:11:03.508398 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0805 23:11:03.508401 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 23:11:03.508557 (+ 156us) server_negotiation.cc:407] Connection header received
0805 23:11:03.508633 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:11:03.508635 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:11:03.508682 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:11:03.508768 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:11:03.509229 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:11:03.509844 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:11:03.510625 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:11:03.510843 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:11:03.510925 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:11:03.513848 (+ 2923us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.92.195:54671: BlockingRecv error: recv got EOF from 127.25.92.195:54671 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 05 23:11:04 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.193@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:11:04 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.193@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
I20260805 23:11:04.121320 2887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:11:04.115644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:37861 (local address 127.25.92.193:51113)
0805 23:11:04.115956 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0805 23:11:04.115970 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0805 23:11:04.116064 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:11:04.116394 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:11:04.116396 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:11:04.116592 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:11:04.116694 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:11:04.116699 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:11:04.117531 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:11:04.117533 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:11:04.118145 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:11:04.118153 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:11:04.118591 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:11:04.118593 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:11:04.118663 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:11:04.119209 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:11:04.119223 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:11:04.121186 (+ 1963us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.92.254:37861: Server oryx/127.25.92.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":111,"threads_started":1}
W20260805 23:11:04.121491 2888 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:11:04.115794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.193:51113 (local address 127.25.92.254:37861)
0805 23:11:04.116019 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0805 23:11:04.116022 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 23:11:04.116035 (+ 13us) server_negotiation.cc:407] Connection header received
0805 23:11:04.116187 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:11:04.116189 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:11:04.116259 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:11:04.116363 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:11:04.116800 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:11:04.117374 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:11:04.118255 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:11:04.118482 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:11:04.118563 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:11:04.121345 (+ 2782us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.92.193:51113: BlockingRecv error: recv got EOF from 127.25.92.193:51113 (error 108)
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":62,"threads_started":1}
Aug 05 23:11:04 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.194@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:11:04 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.194@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
I20260805 23:11:04.308275 2889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:11:04.302087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:37861 (local address 127.25.92.194:35525)
0805 23:11:04.302404 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0805 23:11:04.302419 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0805 23:11:04.302561 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:11:04.302863 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:11:04.302866 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:11:04.303079 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:11:04.303179 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:11:04.303185 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:11:04.303961 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:11:04.303966 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:11:04.304623 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:11:04.304633 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:11:04.305111 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:11:04.305115 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:11:04.305197 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:11:04.305864 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:11:04.305876 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:11:04.308075 (+ 2199us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.92.254:37861: Server oryx/127.25.92.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":105,"threads_started":1}
W20260805 23:11:04.308338 2888 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:11:04.302148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.194:35525 (local address 127.25.92.254:37861)
0805 23:11:04.302364 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0805 23:11:04.302370 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:11:04.302503 (+ 133us) server_negotiation.cc:407] Connection header received
0805 23:11:04.302698 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:11:04.302701 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:11:04.302746 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:11:04.302838 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:11:04.303298 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:11:04.303842 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:11:04.304749 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:11:04.304985 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:11:04.305099 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:11:04.308226 (+ 3127us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.92.194:35525: BlockingRecv error: recv got EOF from 127.25.92.194:35525 (error 108)
Metrics: {"server-negotiator.queue_time_us":109}
Aug 05 23:11:04 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.195@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:11:04 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.195@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
I20260805 23:11:04.521412 2891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:11:04.514793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:37861 (local address 127.25.92.195:44591)
0805 23:11:04.515119 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0805 23:11:04.515151 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0805 23:11:04.515271 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:11:04.515552 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:11:04.515555 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:11:04.515789 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:11:04.515886 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:11:04.515892 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:11:04.516699 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:11:04.516705 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:11:04.517366 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:11:04.517376 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:11:04.517994 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:11:04.517998 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:11:04.518085 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:11:04.518851 (+ 766us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:11:04.518863 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:11:04.521245 (+ 2382us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.92.254:37861: Server oryx/127.25.92.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":93,"threads_started":1}
W20260805 23:11:04.521562 2888 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:11:04.514982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.195:44591 (local address 127.25.92.254:37861)
0805 23:11:04.515264 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0805 23:11:04.515269 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:11:04.515285 (+ 16us) server_negotiation.cc:407] Connection header received
0805 23:11:04.515369 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:11:04.515372 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:11:04.515432 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:11:04.515528 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:11:04.516017 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:11:04.516568 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:11:04.517537 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:11:04.517868 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:11:04.517986 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:11:04.521398 (+ 3412us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.92.195:44591: BlockingRecv error: recv got EOF from 127.25.92.195:44591 (error 108)
Metrics: {"server-negotiator.queue_time_us":140}
Aug 05 23:11:05 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.193@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:11:05 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.193@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
I20260805 23:11:05.128262 2892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:11:05.122426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:37861 (local address 127.25.92.193:35513)
0805 23:11:05.122784 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0805 23:11:05.122799 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0805 23:11:05.122907 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:11:05.123154 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:11:05.123156 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:11:05.123339 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:11:05.123435 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:11:05.123440 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:11:05.124333 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:11:05.124338 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:11:05.124928 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:11:05.124936 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:11:05.125433 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:11:05.125436 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:11:05.125566 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:11:05.126161 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:11:05.126174 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:11:05.128136 (+ 1962us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.92.254:37861: Server oryx/127.25.92.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":79,"threads_started":1}
W20260805 23:11:05.128460 2893 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:11:05.122587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.193:35513 (local address 127.25.92.254:37861)
0805 23:11:05.122859 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0805 23:11:05.122863 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:11:05.122878 (+ 15us) server_negotiation.cc:407] Connection header received
0805 23:11:05.122985 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:11:05.122987 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:11:05.123032 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:11:05.123113 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:11:05.123577 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:11:05.124193 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:11:05.125058 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:11:05.125303 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:11:05.125429 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:11:05.128308 (+ 2879us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.92.193:35513: BlockingRecv error: recv got EOF from 127.25.92.193:35513 (error 108)
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":87,"threads_started":1}
Aug 05 23:11:05 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.194@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:11:05 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.194@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
I20260805 23:11:05.315275 2894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:11:05.309139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:37861 (local address 127.25.92.194:53671)
0805 23:11:05.309542 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0805 23:11:05.309563 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0805 23:11:05.309662 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:11:05.309956 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:11:05.309958 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:11:05.310147 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:11:05.310248 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:11:05.310253 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:11:05.311133 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:11:05.311136 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:11:05.311740 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:11:05.311750 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:11:05.312255 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:11:05.312259 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:11:05.312327 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:11:05.312965 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:11:05.312978 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:11:05.315095 (+ 2117us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.92.254:37861: Server oryx/127.25.92.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":196,"threads_started":1}
W20260805 23:11:05.315378 2893 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:11:05.309288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.194:53671 (local address 127.25.92.254:37861)
0805 23:11:05.309449 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0805 23:11:05.309453 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:11:05.309660 (+ 207us) server_negotiation.cc:407] Connection header received
0805 23:11:05.309790 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:11:05.309792 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:11:05.309842 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:11:05.309916 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:11:05.310393 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:11:05.310987 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:11:05.311880 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:11:05.312113 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:11:05.312197 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:11:05.315262 (+ 3065us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.92.194:53671: BlockingRecv error: recv got EOF from 127.25.92.194:53671 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 05 23:11:05 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.195@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:11:05 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.195@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
I20260805 23:11:05.531837 2895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:11:05.522327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:37861 (local address 127.25.92.195:59311)
0805 23:11:05.522647 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0805 23:11:05.522673 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0805 23:11:05.522819 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:11:05.523098 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:11:05.523101 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:11:05.523316 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:11:05.523408 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:11:05.523414 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:11:05.524223 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:11:05.524226 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:11:05.524781 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:11:05.524789 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:11:05.525282 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:11:05.525286 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:11:05.525368 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:11:05.529439 (+ 4071us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:11:05.529454 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:11:05.531641 (+ 2187us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.92.254:37861: Server oryx/127.25.92.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":86,"threads_started":1}
W20260805 23:11:05.531929 2893 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:11:05.522473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.195:59311 (local address 127.25.92.254:37861)
0805 23:11:05.522635 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0805 23:11:05.522640 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:11:05.522756 (+ 116us) server_negotiation.cc:407] Connection header received
0805 23:11:05.522924 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:11:05.522927 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:11:05.522983 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:11:05.523072 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:11:05.523535 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:11:05.524087 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:11:05.524904 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:11:05.525139 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:11:05.525281 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:11:05.531803 (+ 6522us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.92.195:59311: BlockingRecv error: recv got EOF from 127.25.92.195:59311 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 05 23:11:06 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.193@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:11:06 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.193@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
I20260805 23:11:06.135386 2896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:11:06.129266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:37861 (local address 127.25.92.193:45859)
0805 23:11:06.129639 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0805 23:11:06.129654 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0805 23:11:06.129758 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:11:06.130010 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:11:06.130012 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:11:06.130214 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:11:06.130313 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:11:06.130319 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:11:06.131111 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:11:06.131114 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:11:06.131786 (+ 672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:11:06.131795 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:11:06.132257 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:11:06.132260 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:11:06.132337 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:11:06.132997 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:11:06.133009 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:11:06.135260 (+ 2251us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.92.254:37861: Server oryx/127.25.92.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":100,"threads_started":1}
W20260805 23:11:06.135672 2897 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:11:06.129336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.193:45859 (local address 127.25.92.254:37861)
0805 23:11:06.129658 (+ 322us) server_negotiation.cc:206] Beginning negotiation
0805 23:11:06.129662 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:11:06.129726 (+ 64us) server_negotiation.cc:407] Connection header received
0805 23:11:06.129840 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:11:06.129842 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:11:06.129887 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:11:06.129968 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:11:06.130422 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:11:06.130999 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:11:06.131897 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:11:06.132118 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:11:06.132197 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:11:06.135459 (+ 3262us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.92.193:45859: BlockingRecv error: recv got EOF from 127.25.92.193:45859 (error 108)
Metrics: {"server-negotiator.queue_time_us":226,"thread_start_us":91,"threads_started":1}
Aug 05 23:11:06 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.194@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:11:06 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.194@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
I20260805 23:11:06.322201 2898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:11:06.316285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:37861 (local address 127.25.92.194:48141)
0805 23:11:06.316653 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0805 23:11:06.316670 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0805 23:11:06.316795 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:11:06.317081 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:11:06.317085 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:11:06.317302 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:11:06.317393 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:11:06.317398 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:11:06.318250 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:11:06.318255 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:11:06.318820 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:11:06.318828 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:11:06.319322 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:11:06.319326 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:11:06.319417 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:11:06.320052 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:11:06.320064 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:11:06.322009 (+ 1945us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.92.254:37861: Server oryx/127.25.92.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":118,"threads_started":1}
W20260805 23:11:06.322295 2897 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:11:06.316519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.194:48141 (local address 127.25.92.254:37861)
0805 23:11:06.316703 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0805 23:11:06.316710 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0805 23:11:06.316726 (+ 16us) server_negotiation.cc:407] Connection header received
0805 23:11:06.316907 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:11:06.316911 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:11:06.316964 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:11:06.317058 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:11:06.317560 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:11:06.318131 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:11:06.318955 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:11:06.319194 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:11:06.319318 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:11:06.322158 (+ 2840us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.92.194:48141: BlockingRecv error: recv got EOF from 127.25.92.194:48141 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
Aug 05 23:11:06 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.195@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:11:06 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.195@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
I20260805 23:11:06.538703 2899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:11:06.532782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:37861 (local address 127.25.92.195:45877)
0805 23:11:06.533077 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0805 23:11:06.533094 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0805 23:11:06.533249 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:11:06.533670 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:11:06.533673 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:11:06.533861 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:11:06.533962 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:11:06.533968 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:11:06.534774 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:11:06.534777 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:11:06.535325 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:11:06.535332 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:11:06.535795 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:11:06.535798 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:11:06.535860 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:11:06.536428 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:11:06.536439 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:11:06.538554 (+ 2115us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.92.254:37861: Server oryx/127.25.92.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":114,"threads_started":1}
W20260805 23:11:06.538882 2897 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:11:06.532881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.195:45877 (local address 127.25.92.254:37861)
0805 23:11:06.533050 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0805 23:11:06.533054 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:11:06.533246 (+ 192us) server_negotiation.cc:407] Connection header received
0805 23:11:06.533335 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:11:06.533338 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:11:06.533395 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:11:06.533516 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:11:06.534100 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:11:06.534656 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:11:06.535451 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:11:06.535690 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:11:06.535772 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:11:06.538726 (+ 2954us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.92.195:45877: BlockingRecv error: recv got EOF from 127.25.92.195:45877 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 05 23:11:07 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.193@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:11:07 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.193@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
I20260805 23:11:07.143778 2900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:11:07.136313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:37861 (local address 127.25.92.193:49389)
0805 23:11:07.136627 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0805 23:11:07.136642 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0805 23:11:07.136759 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:11:07.137186 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:11:07.137189 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:11:07.137378 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:11:07.137535 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:11:07.137542 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:11:07.138501 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:11:07.138504 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:11:07.139077 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:11:07.139086 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:11:07.139689 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:11:07.139692 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:11:07.139764 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:11:07.140390 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:11:07.140403 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:11:07.143649 (+ 3246us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.92.254:37861: Server oryx/127.25.92.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":101,"threads_started":1}
W20260805 23:11:07.144188 2901 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:11:07.136394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.193:49389 (local address 127.25.92.254:37861)
0805 23:11:07.136737 (+ 343us) server_negotiation.cc:206] Beginning negotiation
0805 23:11:07.136741 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:11:07.136756 (+ 15us) server_negotiation.cc:407] Connection header received
0805 23:11:07.136953 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:11:07.136956 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:11:07.137015 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:11:07.137109 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:11:07.137729 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:11:07.138360 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:11:07.139266 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:11:07.139507 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:11:07.139609 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:11:07.143842 (+ 4233us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.92.193:49389: BlockingRecv error: recv got EOF from 127.25.92.193:49389 (error 108)
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":69,"threads_started":1}
Aug 05 23:11:07 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.194@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:11:07 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.194@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
I20260805 23:11:07.329300 2902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:11:07.323246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:37861 (local address 127.25.92.194:39229)
0805 23:11:07.323604 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0805 23:11:07.323621 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0805 23:11:07.323823 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:11:07.324123 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:11:07.324126 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:11:07.324360 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:11:07.324459 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:11:07.324465 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:11:07.325265 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:11:07.325269 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:11:07.325939 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:11:07.325949 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:11:07.326440 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:11:07.326444 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:11:07.326528 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:11:07.327126 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:11:07.327138 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:11:07.329091 (+ 1953us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.92.254:37861: Server oryx/127.25.92.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":126,"threads_started":1}
W20260805 23:11:07.329420 2901 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:11:07.323410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.194:39229 (local address 127.25.92.254:37861)
0805 23:11:07.323583 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0805 23:11:07.323587 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:11:07.323750 (+ 163us) server_negotiation.cc:407] Connection header received
0805 23:11:07.323949 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:11:07.323952 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:11:07.323998 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:11:07.324097 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:11:07.324588 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:11:07.325131 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:11:07.326068 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:11:07.326312 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:11:07.326434 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:11:07.329280 (+ 2846us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.92.194:39229: BlockingRecv error: recv got EOF from 127.25.92.194:39229 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 05 23:11:07 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.195@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:11:07 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.195@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
I20260805 23:11:07.546892 2903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:11:07.539922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:37861 (local address 127.25.92.195:37137)
0805 23:11:07.540343 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0805 23:11:07.540360 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0805 23:11:07.540526 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:11:07.540863 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:11:07.540866 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:11:07.541170 (+ 304us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:11:07.541326 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:11:07.541334 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:11:07.542545 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:11:07.542549 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:11:07.543352 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:11:07.543363 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:11:07.543862 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:11:07.543864 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:11:07.543937 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:11:07.544608 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:11:07.544632 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:11:07.546751 (+ 2119us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.92.254:37861: Server oryx/127.25.92.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":104,"threads_started":1}
W20260805 23:11:07.547097 2901 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:11:07.540169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.195:37137 (local address 127.25.92.254:37861)
0805 23:11:07.540339 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0805 23:11:07.540344 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:11:07.540458 (+ 114us) server_negotiation.cc:407] Connection header received
0805 23:11:07.540649 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:11:07.540652 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:11:07.540712 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:11:07.540809 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:11:07.541440 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:11:07.542402 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:11:07.543486 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:11:07.543744 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:11:07.543831 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:11:07.546925 (+ 3094us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.92.195:37137: BlockingRecv error: recv got EOF from 127.25.92.195:37137 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 05 23:11:08 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.193@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:11:08 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.193@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
I20260805 23:11:08.151386 2904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:11:08.144930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:37861 (local address 127.25.92.193:39689)
0805 23:11:08.145263 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0805 23:11:08.145278 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0805 23:11:08.145380 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:11:08.145694 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:11:08.145696 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:11:08.145899 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:11:08.145999 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:11:08.146004 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:11:08.146827 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:11:08.146829 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:11:08.147412 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:11:08.147421 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:11:08.147883 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:11:08.147886 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:11:08.147951 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:11:08.148589 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:11:08.148653 (+ 64us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:11:08.151209 (+ 2556us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.92.254:37861: Server oryx/127.25.92.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":114,"threads_started":1}
W20260805 23:11:08.151479 2905 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:11:08.145131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.193:39689 (local address 127.25.92.254:37861)
0805 23:11:08.145393 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0805 23:11:08.145397 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:11:08.145413 (+ 16us) server_negotiation.cc:407] Connection header received
0805 23:11:08.145458 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:11:08.145460 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:11:08.145527 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:11:08.145608 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:11:08.146132 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:11:08.146711 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:11:08.147519 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:11:08.147773 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:11:08.147855 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:11:08.151345 (+ 3490us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.92.193:39689: BlockingRecv error: recv got EOF from 127.25.92.193:39689 (error 108)
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":103,"threads_started":1}
Aug 05 23:11:08 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.194@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:11:08 dist-test-slave-ss4n krb5kdc[1806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.92.194@KRBTEST.COM for oryx/127.25.92.254@KRBTEST.COM, Server not found in Kerberos database
I20260805 23:11:08.337792 2906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:11:08.330556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:37861 (local address 127.25.92.194:57369)
0805 23:11:08.330887 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0805 23:11:08.330903 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0805 23:11:08.331027 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:11:08.331404 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:11:08.331407 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:11:08.331666 (+ 259us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:11:08.331777 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:11:08.331783 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:11:08.332683 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:11:08.332686 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:11:08.333280 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:11:08.333290 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:11:08.333871 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:11:08.333875 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:11:08.333962 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:11:08.334905 (+ 943us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:11:08.334919 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:11:08.337586 (+ 2667us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.92.254:37861: Server oryx/127.25.92.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":128,"threads_started":1}
W20260805 23:11:08.337888 2905 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:11:08.330737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.194:57369 (local address 127.25.92.254:37861)
0805 23:11:08.330909 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0805 23:11:08.330913 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:11:08.330972 (+ 59us) server_negotiation.cc:407] Connection header received
0805 23:11:08.331172 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:11:08.331176 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:11:08.331228 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:11:08.331318 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:11:08.331924 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:11:08.332478 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:11:08.333430 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:11:08.333742 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:11:08.333831 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:11:08.337756 (+ 3925us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.92.194:57369: BlockingRecv error: recv got EOF from 127.25.92.194:57369 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
I20260805 23:11:08.412261 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 2411
I20260805 23:11:08.418778 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 2570
I20260805 23:11:08.427495 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 2706
I20260805 23:11:08.433941 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 2339
2026-08-05T23:11:08Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18142 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-05T23:11:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:11:08Z Disabled control of system clock
I20260805 23:11:08.468521 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971406251336-25971-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971406251336-25971-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971406251336-25971-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.254:40877
--webserver_interface=127.25.92.254
--webserver_port=0
--builtin_ntp_servers=127.25.92.212:43493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.92.254:40877
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260805 23:11:08.588964 2910 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:11:08.589268 2910 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:11:08.589304 2910 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:11:08.593457 2910 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:11:08.593567 2910 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:11:08.593583 2910 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:11:08.593601 2910 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:11:08.593617 2910 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:11:08.598582 2910 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:43493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971406251336-25971-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971406251336-25971-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.92.254:40877
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.254:40877
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.92.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2910
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971406251336-25971-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:11:08.599674 2910 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:11:08.601037 2910 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:11:08.610225 2916 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:11:08.610222 2915 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:11:08.610365 2919 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:11:08.610419 2910 server_base.cc:1034] running on GCE node
I20260805 23:11:08.611305 2910 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:11:08.612457 2910 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:11:08.613590 2910 hybrid_clock.cc:648] HybridClock initialized: now 1785971468613556 us; error 55 us; skew 500 ppm
I20260805 23:11:08.615903 2910 webserver.cc:460] Webserver started at http://127.25.92.254:39999/ using document root <none> and password file <none>
I20260805 23:11:08.616577 2910 fs_manager.cc:359] Metadata directory not provided
I20260805 23:11:08.616652 2910 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:11:08.616901 2910 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:11:08.619271 2910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971406251336-25971-0/minicluster-data/master-0/data/instance:
uuid: "48a1cc93f9e64aeb9a86c44bfe6ee940"
format_stamp: "Formatted at 2026-08-05 23:11:08 on dist-test-slave-ss4n"
server_key: "03e527403c6a56e08134e9f9137e1eed"
server_key_iv: "86090358bd1aa91523d229027890de29"
server_key_version: "encryptionkey@0"
I20260805 23:11:08.619910 2910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971406251336-25971-0/minicluster-data/master-0/wal/instance:
uuid: "48a1cc93f9e64aeb9a86c44bfe6ee940"
format_stamp: "Formatted at 2026-08-05 23:11:08 on dist-test-slave-ss4n"
server_key: "03e527403c6a56e08134e9f9137e1eed"
server_key_iv: "86090358bd1aa91523d229027890de29"
server_key_version: "encryptionkey@0"
I20260805 23:11:08.624571 2910 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260805 23:11:08.627434 2926 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:11:08.628911 2910 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260805 23:11:08.629061 2910 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971406251336-25971-0/minicluster-data/master-0/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971406251336-25971-0/minicluster-data/master-0/wal
uuid: "48a1cc93f9e64aeb9a86c44bfe6ee940"
format_stamp: "Formatted at 2026-08-05 23:11:08 on dist-test-slave-ss4n"
server_key: "03e527403c6a56e08134e9f9137e1eed"
server_key_iv: "86090358bd1aa91523d229027890de29"
server_key_version: "encryptionkey@0"
I20260805 23:11:08.629338 2910 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971406251336-25971-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971406251336-25971-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971406251336-25971-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:11:08.646131 2910 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:11:08.646889 2910 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:11:08.647060 2910 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:11:08.655068 2910 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.254:40877
I20260805 23:11:08.655100 2981 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.254:40877 every 8 connection(s)
I20260805 23:11:08.656134 2910 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
I20260805 23:11:08.659771 2982 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:11:08.665831 2982 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 48a1cc93f9e64aeb9a86c44bfe6ee940: Bootstrap starting.
I20260805 23:11:08.666069 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 2910
I20260805 23:11:08.666200 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971406251336-25971-0/minicluster-data/master-0/wal/instance
I20260805 23:11:08.666499 25971 external_mini_cluster.cc:1428] Setting key 29cf0d6a16407ccaab1ec3d3395434c7
I20260805 23:11:08.669085 2982 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 48a1cc93f9e64aeb9a86c44bfe6ee940: Neither blocks nor log segments found. Creating new log.
I20260805 23:11:08.669991 2982 log.cc:824] T 00000000000000000000000000000000 P 48a1cc93f9e64aeb9a86c44bfe6ee940: Log is configured to *not* fsync() on all Append() calls
I20260805 23:11:08.672247 2982 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 48a1cc93f9e64aeb9a86c44bfe6ee940: No bootstrap required, opened a new log
I20260805 23:11:08.676750 2982 raft_consensus.cc:348] T 00000000000000000000000000000000 P 48a1cc93f9e64aeb9a86c44bfe6ee940 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48a1cc93f9e64aeb9a86c44bfe6ee940" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 40877 } }
I20260805 23:11:08.677035 2982 raft_consensus.cc:374] T 00000000000000000000000000000000 P 48a1cc93f9e64aeb9a86c44bfe6ee940 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:11:08.677112 2982 raft_consensus.cc:729] T 00000000000000000000000000000000 P 48a1cc93f9e64aeb9a86c44bfe6ee940 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48a1cc93f9e64aeb9a86c44bfe6ee940, State: Initialized, Role: FOLLOWER
I20260805 23:11:08.677613 2985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:11:08.668025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57094 (local address 127.25.92.254:40877)
0805 23:11:08.668888 (+ 863us) server_negotiation.cc:206] Beginning negotiation
0805 23:11:08.668903 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0805 23:11:08.668942 (+ 39us) server_negotiation.cc:407] Connection header received
0805 23:11:08.669794 (+ 852us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:11:08.669814 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:11:08.670287 (+ 473us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:11:08.670635 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:11:08.671125 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:11:08.672371 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:11:08.673363 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:11:08.673958 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:11:08.674145 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:11:08.675159 (+ 1014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:11:08.675165 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:11:08.675185 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:11:08.675770 (+ 585us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:11:08.675846 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:11:08.675873 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:11:08.676016 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0805 23:11:08.676229 (+ 213us) server_negotiation.cc:299] Negotiation successful
0805 23:11:08.676669 (+ 440us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":473,"thread_start_us":174,"threads_started":1}
I20260805 23:11:08.677695 2982 consensus_queue.cc:260] T 00000000000000000000000000000000 P 48a1cc93f9e64aeb9a86c44bfe6ee940 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48a1cc93f9e64aeb9a86c44bfe6ee940" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 40877 } }
I20260805 23:11:08.677923 2982 raft_consensus.cc:388] T 00000000000000000000000000000000 P 48a1cc93f9e64aeb9a86c44bfe6ee940 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:11:08.678181 2982 raft_consensus.cc:482] T 00000000000000000000000000000000 P 48a1cc93f9e64aeb9a86c44bfe6ee940 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:11:08.678308 2982 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 48a1cc93f9e64aeb9a86c44bfe6ee940 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:11:08.679821 2982 raft_consensus.cc:504] T 00000000000000000000000000000000 P 48a1cc93f9e64aeb9a86c44bfe6ee940 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48a1cc93f9e64aeb9a86c44bfe6ee940" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 40877 } }
I20260805 23:11:08.680280 2982 leader_election.cc:302] T 00000000000000000000000000000000 P 48a1cc93f9e64aeb9a86c44bfe6ee940 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 48a1cc93f9e64aeb9a86c44bfe6ee940; no voters:
I20260805 23:11:08.680696 2982 leader_election.cc:288] T 00000000000000000000000000000000 P 48a1cc93f9e64aeb9a86c44bfe6ee940 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 23:11:08.680923 2987 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 48a1cc93f9e64aeb9a86c44bfe6ee940 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:11:08.681218 2987 raft_consensus.cc:686] T 00000000000000000000000000000000 P 48a1cc93f9e64aeb9a86c44bfe6ee940 [term 1 LEADER]: Becoming Leader. State: Replica: 48a1cc93f9e64aeb9a86c44bfe6ee940, State: Running, Role: LEADER
I20260805 23:11:08.681807 2987 consensus_queue.cc:237] T 00000000000000000000000000000000 P 48a1cc93f9e64aeb9a86c44bfe6ee940 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48a1cc93f9e64aeb9a86c44bfe6ee940" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 40877 } }
I20260805 23:11:08.682293 2982 sys_catalog.cc:564] T 00000000000000000000000000000000 P 48a1cc93f9e64aeb9a86c44bfe6ee940 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:11:08.683737 2989 sys_catalog.cc:455] T 00000000000000000000000000000000 P 48a1cc93f9e64aeb9a86c44bfe6ee940 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 48a1cc93f9e64aeb9a86c44bfe6ee940. Latest consensus state: current_term: 1 leader_uuid: "48a1cc93f9e64aeb9a86c44bfe6ee940" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48a1cc93f9e64aeb9a86c44bfe6ee940" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 40877 } } }
I20260805 23:11:08.683904 2989 sys_catalog.cc:458] T 00000000000000000000000000000000 P 48a1cc93f9e64aeb9a86c44bfe6ee940 [sys.catalog]: This master's current role is: LEADER
I20260805 23:11:08.683737 2988 sys_catalog.cc:455] T 00000000000000000000000000000000 P 48a1cc93f9e64aeb9a86c44bfe6ee940 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "48a1cc93f9e64aeb9a86c44bfe6ee940" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48a1cc93f9e64aeb9a86c44bfe6ee940" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 40877 } } }
I20260805 23:11:08.684233 2988 sys_catalog.cc:458] T 00000000000000000000000000000000 P 48a1cc93f9e64aeb9a86c44bfe6ee940 [sys.catalog]: This master's current role is: LEADER
I20260805 23:11:08.684585 2997 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:11:08.688131 2997 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:11:08.694280 2997 catalog_manager.cc:1347] Generated new cluster ID: 5737e4ce4b3f4d82bd846f4dfccc7625
I20260805 23:11:08.694378 2997 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:11:08.703769 2997 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:11:08.705184 2997 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:11:08.713896 2997 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 48a1cc93f9e64aeb9a86c44bfe6ee940: Generated new TSK 0
I20260805 23:11:08.714872 2997 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260805 23:11:08.728879 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971406251336-25971-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971406251336-25971-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971406251336-25971-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971406251336-25971-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.92.193:0
--local_ip_for_outbound_sockets=127.25.92.193
--webserver_interface=127.25.92.193
--webserver_port=0
--tserver_master_addrs=127.25.92.254:40877
--builtin_ntp_servers=127.25.92.212:43493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260805 23:11:08.848042 3007 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:11:08.848371 3007 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:11:08.848448 3007 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:11:08.852643 3007 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:11:08.852754 3007 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:11:08.852849 3007 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.92.193
I20260805 23:11:08.857986 3007 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:43493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971406251336-25971-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971406251336-25971-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.92.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971406251336-25971-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.92.193
--webserver_port=0
--tserver_master_addrs=127.25.92.254:40877
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3007
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.92.193
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971406251336-25971-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:11:08.859197 3007 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:11:08.860673 3007 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:11:08.871878 3013 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:11:08.871985 3016 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:11:08.871878 3012 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:11:08.872337 3007 server_base.cc:1034] running on GCE node
I20260805 23:11:08.872867 3007 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:11:08.873711 3007 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:11:08.874925 3007 hybrid_clock.cc:648] HybridClock initialized: now 1785971468874882 us; error 61 us; skew 500 ppm
I20260805 23:11:08.877087 3007 webserver.cc:460] Webserver started at http://127.25.92.193:42707/ using document root <none> and password file <none>
I20260805 23:11:08.877880 3007 fs_manager.cc:359] Metadata directory not provided
I20260805 23:11:08.877990 3007 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:11:08.878247 3007 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:11:08.880595 3007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971406251336-25971-0/minicluster-data/ts-0/data/instance:
uuid: "be548f5e5c494b74a77034eab79bfe12"
format_stamp: "Formatted at 2026-08-05 23:11:08 on dist-test-slave-ss4n"
server_key: "a3b3142aedae12ebe896d949a58b61a6"
server_key_iv: "6018eda2829491380341b089fcf49e42"
server_key_version: "encryptionkey@0"
I20260805 23:11:08.881242 3007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971406251336-25971-0/minicluster-data/ts-0/wal/instance:
uuid: "be548f5e5c494b74a77034eab79bfe12"
format_stamp: "Formatted at 2026-08-05 23:11:08 on dist-test-slave-ss4n"
server_key: "a3b3142aedae12ebe896d949a58b61a6"
server_key_iv: "6018eda2829491380341b089fcf49e42"
server_key_version: "encryptionkey@0"
I20260805 23:11:08.886184 3007 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260805 23:11:08.889125 3022 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:11:08.890861 3007 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:11:08.891042 3007 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971406251336-25971-0/minicluster-data/ts-0/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971406251336-25971-0/minicluster-data/ts-0/wal
uuid: "be548f5e5c494b74a77034eab79bfe12"
format_stamp: "Formatted at 2026-08-05 23:11:08 on dist-test-slave-ss4n"
server_key: "a3b3142aedae12ebe896d949a58b61a6"
server_key_iv: "6018eda2829491380341b089fcf49e42"
server_key_version: "encryptionkey@0"
I20260805 23:11:08.891207 3007 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971406251336-25971-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971406251336-25971-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971406251336-25971-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:11:08.925961 3007 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:11:08.926837 3007 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:11:08.927083 3007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:11:08.927775 3007 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:11:08.928877 3007 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:11:08.928963 3007 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:11:08.929044 3007 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:11:08.929093 3007 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:11:08.940780 3007 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.193:45673
I20260805 23:11:08.940850 3139 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.193:45673 every 8 connection(s)
I20260805 23:11:08.941901 3007 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971406251336-25971-0/minicluster-data/ts-0/data/info.pb
I20260805 23:11:08.947149 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 3007
I20260805 23:11:08.947288 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971406251336-25971-0/minicluster-data/ts-0/wal/instance
I20260805 23:11:08.947633 25971 external_mini_cluster.cc:1428] Setting key 89993e00c78438c1c2bcf3638fa14b8c
I20260805 23:11:08.949965 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971406251336-25971-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971406251336-25971-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971406251336-25971-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971406251336-25971-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.92.194:0
--local_ip_for_outbound_sockets=127.25.92.194
--webserver_interface=127.25.92.194
--webserver_port=0
--tserver_master_addrs=127.25.92.254:40877
--builtin_ntp_servers=127.25.92.212:43493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260805 23:11:08.953809 2985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:11:08.944125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.193:57075 (local address 127.25.92.254:40877)
0805 23:11:08.944298 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0805 23:11:08.944303 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:11:08.945718 (+ 1415us) server_negotiation.cc:407] Connection header received
0805 23:11:08.947197 (+ 1479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:11:08.947201 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:11:08.947280 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:11:08.947401 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:11:08.948633 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:11:08.949432 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:11:08.950855 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:11:08.951338 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:11:08.951496 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:11:08.953042 (+ 1546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:11:08.953049 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:11:08.953056 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:11:08.953077 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:11:08.953106 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:11:08.953121 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:11:08.953222 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0805 23:11:08.953423 (+ 201us) server_negotiation.cc:299] Negotiation successful
0805 23:11:08.953555 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260805 23:11:08.954692 3142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:11:08.944554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:40877 (local address 127.25.92.193:57075)
0805 23:11:08.945356 (+ 802us) negotiation.cc:107] Waiting for socket to connect
0805 23:11:08.945416 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0805 23:11:08.946919 (+ 1503us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:11:08.947591 (+ 672us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:11:08.947599 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:11:08.947690 (+ 91us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:11:08.948418 (+ 728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:11:08.948464 (+ 46us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:11:08.949625 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:11:08.949628 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:11:08.950717 (+ 1089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:11:08.950728 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:11:08.951616 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:11:08.951623 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:11:08.951868 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:11:08.952660 (+ 792us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0805 23:11:08.952662 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:11:08.952723 (+ 61us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0805 23:11:08.952818 (+ 95us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:11:08.953246 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:11:08.953255 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:11:08.953261 (+ 6us) client_negotiation.cc:769] Sending connection context
0805 23:11:08.953387 (+ 126us) client_negotiation.cc:240] Negotiation successful
0805 23:11:08.953783 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":144,"threads_started":1}
I20260805 23:11:08.956255 3140 heartbeater.cc:347] Connected to a master server at 127.25.92.254:40877
I20260805 23:11:08.956620 3140 heartbeater.cc:464] Registering TS with master...
I20260805 23:11:08.957255 3140 heartbeater.cc:511] Master 127.25.92.254:40877 requested a full tablet report, sending...
I20260805 23:11:08.959182 2944 ts_manager.cc:194] Registered new tserver with Master: be548f5e5c494b74a77034eab79bfe12 (127.25.92.193:45673)
I20260805 23:11:08.960405 2944 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.25.92.193:57075
W20260805 23:11:09.072769 3143 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:11:09.073050 3143 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:11:09.073088 3143 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:11:09.077409 3143 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:11:09.077492 3143 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:11:09.077551 3143 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.92.194
I20260805 23:11:09.082783 3143 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:43493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971406251336-25971-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971406251336-25971-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.92.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971406251336-25971-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.92.194
--webserver_port=0
--tserver_master_addrs=127.25.92.254:40877
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3143
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.92.194
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971406251336-25971-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:11:09.084059 3143 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:11:09.085602 3143 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:11:09.096177 3148 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:11:09.096185 3149 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:11:09.096185 3152 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:11:09.096446 3143 server_base.cc:1034] running on GCE node
I20260805 23:11:09.096897 3143 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:11:09.097847 3143 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:11:09.099062 3143 hybrid_clock.cc:648] HybridClock initialized: now 1785971469099031 us; error 61 us; skew 500 ppm
I20260805 23:11:09.101209 3143 webserver.cc:460] Webserver started at http://127.25.92.194:36917/ using document root <none> and password file <none>
I20260805 23:11:09.101912 3143 fs_manager.cc:359] Metadata directory not provided
I20260805 23:11:09.102025 3143 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:11:09.102303 3143 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:11:09.104559 3143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971406251336-25971-0/minicluster-data/ts-1/data/instance:
uuid: "921ead44bb464a18b030aab442766e07"
format_stamp: "Formatted at 2026-08-05 23:11:09 on dist-test-slave-ss4n"
server_key: "68667253279ed1e3d27e65bcd9e24120"
server_key_iv: "cda7beec4c95757f2ac6efedf975c24c"
server_key_version: "encryptionkey@0"
I20260805 23:11:09.105202 3143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971406251336-25971-0/minicluster-data/ts-1/wal/instance:
uuid: "921ead44bb464a18b030aab442766e07"
format_stamp: "Formatted at 2026-08-05 23:11:09 on dist-test-slave-ss4n"
server_key: "68667253279ed1e3d27e65bcd9e24120"
server_key_iv: "cda7beec4c95757f2ac6efedf975c24c"
server_key_version: "encryptionkey@0"
I20260805 23:11:09.109432 3143 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260805 23:11:09.112231 3158 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:11:09.113705 3143 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:11:09.113840 3143 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971406251336-25971-0/minicluster-data/ts-1/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971406251336-25971-0/minicluster-data/ts-1/wal
uuid: "921ead44bb464a18b030aab442766e07"
format_stamp: "Formatted at 2026-08-05 23:11:09 on dist-test-slave-ss4n"
server_key: "68667253279ed1e3d27e65bcd9e24120"
server_key_iv: "cda7beec4c95757f2ac6efedf975c24c"
server_key_version: "encryptionkey@0"
I20260805 23:11:09.113965 3143 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971406251336-25971-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971406251336-25971-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971406251336-25971-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:11:09.127767 3143 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:11:09.128556 3143 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:11:09.128716 3143 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:11:09.129297 3143 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:11:09.130323 3143 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:11:09.130370 3143 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:11:09.130410 3143 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:11:09.130426 3143 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:11:09.139627 3143 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.194:42275
I20260805 23:11:09.139653 3282 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.194:42275 every 8 connection(s)
I20260805 23:11:09.140581 3143 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971406251336-25971-0/minicluster-data/ts-1/data/info.pb
I20260805 23:11:09.148550 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 3143
I20260805 23:11:09.148649 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971406251336-25971-0/minicluster-data/ts-1/wal/instance
I20260805 23:11:09.148993 25971 external_mini_cluster.cc:1428] Setting key 424c58790db4fbc9f8544f96f3c86b0a
I20260805 23:11:09.151366 2985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:11:09.142703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.194:53073 (local address 127.25.92.254:40877)
0805 23:11:09.142865 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0805 23:11:09.142869 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:11:09.144070 (+ 1201us) server_negotiation.cc:407] Connection header received
0805 23:11:09.145661 (+ 1591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:11:09.145664 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:11:09.145730 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:11:09.145816 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:11:09.146810 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:11:09.147370 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:11:09.148582 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:11:09.148915 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:11:09.149049 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:11:09.150437 (+ 1388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:11:09.150442 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:11:09.150445 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:11:09.150458 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:11:09.150484 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:11:09.150496 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:11:09.150569 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0805 23:11:09.150910 (+ 341us) server_negotiation.cc:299] Negotiation successful
0805 23:11:09.151072 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260805 23:11:09.151434 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971406251336-25971-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971406251336-25971-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971406251336-25971-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971406251336-25971-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.92.195:0
--local_ip_for_outbound_sockets=127.25.92.195
--webserver_interface=127.25.92.195
--webserver_port=0
--tserver_master_addrs=127.25.92.254:40877
--builtin_ntp_servers=127.25.92.212:43493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260805 23:11:09.152380 3285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:11:09.143202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:40877 (local address 127.25.92.194:53073)
0805 23:11:09.143902 (+ 700us) negotiation.cc:107] Waiting for socket to connect
0805 23:11:09.143945 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0805 23:11:09.145344 (+ 1399us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:11:09.146003 (+ 659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:11:09.146010 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:11:09.146100 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:11:09.146652 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:11:09.146667 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:11:09.147527 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:11:09.147531 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:11:09.148444 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:11:09.148455 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:11:09.149101 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:11:09.149105 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:11:09.149343 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:11:09.150094 (+ 751us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0805 23:11:09.150096 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:11:09.150160 (+ 64us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0805 23:11:09.150247 (+ 87us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:11:09.150599 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:11:09.150703 (+ 104us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:11:09.150712 (+ 9us) client_negotiation.cc:769] Sending connection context
0805 23:11:09.150859 (+ 147us) client_negotiation.cc:240] Negotiation successful
0805 23:11:09.151191 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":132,"threads_started":1}
I20260805 23:11:09.153740 3283 heartbeater.cc:347] Connected to a master server at 127.25.92.254:40877
I20260805 23:11:09.154038 3283 heartbeater.cc:464] Registering TS with master...
I20260805 23:11:09.154619 3283 heartbeater.cc:511] Master 127.25.92.254:40877 requested a full tablet report, sending...
I20260805 23:11:09.155869 2944 ts_manager.cc:194] Registered new tserver with Master: 921ead44bb464a18b030aab442766e07 (127.25.92.194:42275)
I20260805 23:11:09.156543 2944 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.25.92.194:53073
W20260805 23:11:09.272164 3286 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:11:09.272557 3286 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:11:09.272647 3286 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:11:09.277506 3286 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:11:09.277643 3286 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:11:09.277755 3286 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.92.195
I20260805 23:11:09.284052 3286 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:43493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971406251336-25971-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971406251336-25971-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.92.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971406251336-25971-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.92.195
--webserver_port=0
--tserver_master_addrs=127.25.92.254:40877
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3286
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.92.195
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971406251336-25971-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:11:09.285359 3286 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:11:09.287149 3286 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:11:09.298116 3296 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:11:09.298285 3286 server_base.cc:1034] running on GCE node
W20260805 23:11:09.298116 3292 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:11:09.298476 3294 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:11:09.298913 3286 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:11:09.299630 3286 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:11:09.300870 3286 hybrid_clock.cc:648] HybridClock initialized: now 1785971469300847 us; error 39 us; skew 500 ppm
I20260805 23:11:09.303278 3286 webserver.cc:460] Webserver started at http://127.25.92.195:33517/ using document root <none> and password file <none>
I20260805 23:11:09.304105 3286 fs_manager.cc:359] Metadata directory not provided
I20260805 23:11:09.304224 3286 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:11:09.304490 3286 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:11:09.307036 3286 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971406251336-25971-0/minicluster-data/ts-2/data/instance:
uuid: "4140df07a7124d0d9708c6f05743b966"
format_stamp: "Formatted at 2026-08-05 23:11:09 on dist-test-slave-ss4n"
server_key: "852d21361dbdb0f900b98fb8822c322f"
server_key_iv: "4d24c47bf0b353b590206094daf8530f"
server_key_version: "encryptionkey@0"
I20260805 23:11:09.307701 3286 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971406251336-25971-0/minicluster-data/ts-2/wal/instance:
uuid: "4140df07a7124d0d9708c6f05743b966"
format_stamp: "Formatted at 2026-08-05 23:11:09 on dist-test-slave-ss4n"
server_key: "852d21361dbdb0f900b98fb8822c322f"
server_key_iv: "4d24c47bf0b353b590206094daf8530f"
server_key_version: "encryptionkey@0"
I20260805 23:11:09.312008 3286 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260805 23:11:09.314987 3302 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:11:09.316437 3286 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260805 23:11:09.316598 3286 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971406251336-25971-0/minicluster-data/ts-2/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971406251336-25971-0/minicluster-data/ts-2/wal
uuid: "4140df07a7124d0d9708c6f05743b966"
format_stamp: "Formatted at 2026-08-05 23:11:09 on dist-test-slave-ss4n"
server_key: "852d21361dbdb0f900b98fb8822c322f"
server_key_iv: "4d24c47bf0b353b590206094daf8530f"
server_key_version: "encryptionkey@0"
I20260805 23:11:09.316746 3286 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971406251336-25971-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971406251336-25971-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971406251336-25971-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:11:09.333217 3286 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:11:09.334199 3286 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:11:09.334463 3286 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:11:09.335173 3286 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:11:09.336279 3286 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:11:09.336359 3286 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:11:09.336437 3286 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:11:09.336498 3286 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:11:09.346920 3286 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.195:42019
I20260805 23:11:09.346939 3422 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.195:42019 every 8 connection(s)
I20260805 23:11:09.347930 3286 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971406251336-25971-0/minicluster-data/ts-2/data/info.pb
I20260805 23:11:09.350179 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 3286
I20260805 23:11:09.350294 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971406251336-25971-0/minicluster-data/ts-2/wal/instance
I20260805 23:11:09.350618 25971 external_mini_cluster.cc:1428] Setting key af070b1c37979ad32a93a592a8061805
I20260805 23:11:09.358927 2985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:11:09.350036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.195:56703 (local address 127.25.92.254:40877)
0805 23:11:09.350264 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0805 23:11:09.350271 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0805 23:11:09.351700 (+ 1429us) server_negotiation.cc:407] Connection header received
0805 23:11:09.353708 (+ 2008us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:11:09.353710 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:11:09.353762 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:11:09.353837 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:11:09.354833 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:11:09.355393 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:11:09.356704 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:11:09.356939 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:11:09.357036 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:11:09.358379 (+ 1343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:11:09.358383 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:11:09.358385 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:11:09.358410 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:11:09.358433 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:11:09.358443 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:11:09.358527 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0805 23:11:09.358721 (+ 194us) server_negotiation.cc:299] Negotiation successful
0805 23:11:09.358789 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260805 23:11:09.359937 3425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:11:09.350521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:40877 (local address 127.25.92.195:56703)
0805 23:11:09.351502 (+ 981us) negotiation.cc:107] Waiting for socket to connect
0805 23:11:09.351568 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0805 23:11:09.353288 (+ 1720us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:11:09.353997 (+ 709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:11:09.354005 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:11:09.354103 (+ 98us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:11:09.354650 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:11:09.354674 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:11:09.355524 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:11:09.355526 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:11:09.356522 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:11:09.356533 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:11:09.357099 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:11:09.357102 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:11:09.357261 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:11:09.358020 (+ 759us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0805 23:11:09.358021 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:11:09.358069 (+ 48us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0805 23:11:09.358147 (+ 78us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:11:09.358537 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:11:09.358545 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:11:09.358550 (+ 5us) client_negotiation.cc:769] Sending connection context
0805 23:11:09.358705 (+ 155us) client_negotiation.cc:240] Negotiation successful
0805 23:11:09.358979 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":169,"threads_started":1}
I20260805 23:11:09.361627 3423 heartbeater.cc:347] Connected to a master server at 127.25.92.254:40877
I20260805 23:11:09.361932 3423 heartbeater.cc:464] Registering TS with master...
I20260805 23:11:09.362563 3423 heartbeater.cc:511] Master 127.25.92.254:40877 requested a full tablet report, sending...
I20260805 23:11:09.363829 2944 ts_manager.cc:194] Registered new tserver with Master: 4140df07a7124d0d9708c6f05743b966 (127.25.92.195:42019)
I20260805 23:11:09.364503 2944 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.25.92.195:56703
I20260805 23:11:09.365453 25971 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260805 23:11:09.369356 2985 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:11:09.368096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57098 (local address 127.25.92.254:40877)
0805 23:11:09.368433 (+ 337us) server_negotiation.cc:206] Beginning negotiation
0805 23:11:09.368438 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:11:09.368600 (+ 162us) server_negotiation.cc:407] Connection header received
0805 23:11:09.368776 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:11:09.368779 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:11:09.368833 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:11:09.368926 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:11:09.369177 (+ 251us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57098: BlockingRecv error: recv got EOF from 127.0.0.1:57098 (error 108)
Metrics: {"server-negotiator.queue_time_us":229}
I20260805 23:11:09.369691 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 3007
I20260805 23:11:09.378697 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 3143
I20260805 23:11:09.385797 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 3286
I20260805 23:11:09.392428 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 2910
2026-08-05T23:11:09Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (955 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-05T23:11:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:11:09Z Disabled control of system clock
I20260805 23:11:09.423621 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971406251336-25971-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971406251336-25971-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971406251336-25971-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.254:40863
--webserver_interface=127.25.92.254
--webserver_port=0
--builtin_ntp_servers=127.25.92.212:40561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.92.254:40863
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260805 23:11:09.545105 3434 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:11:09.545413 3434 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:11:09.545454 3434 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:11:09.549930 3434 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:11:09.550011 3434 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:11:09.550027 3434 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:11:09.550046 3434 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:11:09.550062 3434 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:11:09.555836 3434 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:40561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971406251336-25971-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971406251336-25971-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.92.254:40863
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.254:40863
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.92.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3434
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971406251336-25971-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:11:09.557003 3434 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:11:09.558557 3434 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260805 23:11:09.568210 3434 server_base.cc:1034] running on GCE node
W20260805 23:11:09.568154 3439 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:11:09.568221 3440 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:11:09.568147 3442 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:11:09.569126 3434 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:11:09.570441 3434 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:11:09.571661 3434 hybrid_clock.cc:648] HybridClock initialized: now 1785971469571647 us; error 31 us; skew 500 ppm
I20260805 23:11:09.574064 3434 webserver.cc:460] Webserver started at http://127.25.92.254:33789/ using document root <none> and password file <none>
I20260805 23:11:09.574819 3434 fs_manager.cc:359] Metadata directory not provided
I20260805 23:11:09.574900 3434 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:11:09.575160 3434 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:11:09.577780 3434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971406251336-25971-0/minicluster-data/master-0/data/instance:
uuid: "c6a7bea47f924a079fa6bfe0ad0dca69"
format_stamp: "Formatted at 2026-08-05 23:11:09 on dist-test-slave-ss4n"
server_key: "3d417b7f7d17bf69fefabf6bd7939eae"
server_key_iv: "79bedd90f3ea08276d501f42126f1b61"
server_key_version: "encryptionkey@0"
I20260805 23:11:09.578529 3434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971406251336-25971-0/minicluster-data/master-0/wal/instance:
uuid: "c6a7bea47f924a079fa6bfe0ad0dca69"
format_stamp: "Formatted at 2026-08-05 23:11:09 on dist-test-slave-ss4n"
server_key: "3d417b7f7d17bf69fefabf6bd7939eae"
server_key_iv: "79bedd90f3ea08276d501f42126f1b61"
server_key_version: "encryptionkey@0"
I20260805 23:11:09.583150 3434 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260805 23:11:09.586402 3448 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:11:09.587984 3434 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260805 23:11:09.588181 3434 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971406251336-25971-0/minicluster-data/master-0/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971406251336-25971-0/minicluster-data/master-0/wal
uuid: "c6a7bea47f924a079fa6bfe0ad0dca69"
format_stamp: "Formatted at 2026-08-05 23:11:09 on dist-test-slave-ss4n"
server_key: "3d417b7f7d17bf69fefabf6bd7939eae"
server_key_iv: "79bedd90f3ea08276d501f42126f1b61"
server_key_version: "encryptionkey@0"
I20260805 23:11:09.588340 3434 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971406251336-25971-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971406251336-25971-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971406251336-25971-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:11:09.593894 3434 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:11:09.594691 3434 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:11:09.594913 3434 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:11:09.602504 3434 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.254:40863
I20260805 23:11:09.602521 3502 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.254:40863 every 8 connection(s)
I20260805 23:11:09.603565 3434 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
I20260805 23:11:09.607584 3503 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:11:09.611001 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 3434
I20260805 23:11:09.611151 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971406251336-25971-0/minicluster-data/master-0/wal/instance
I20260805 23:11:09.611474 25971 external_mini_cluster.cc:1428] Setting key 176b5155573d9543d4d09541fdb9b484
I20260805 23:11:09.614861 3503 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6a7bea47f924a079fa6bfe0ad0dca69: Bootstrap starting.
I20260805 23:11:09.617786 3503 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c6a7bea47f924a079fa6bfe0ad0dca69: Neither blocks nor log segments found. Creating new log.
I20260805 23:11:09.618218 3506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:11:09.613214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38230 (local address 127.25.92.254:40863)
0805 23:11:09.614187 (+ 973us) server_negotiation.cc:206] Beginning negotiation
0805 23:11:09.614209 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0805 23:11:09.614247 (+ 38us) server_negotiation.cc:407] Connection header received
0805 23:11:09.615006 (+ 759us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:11:09.615040 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:11:09.615408 (+ 368us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:11:09.615761 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:11:09.616666 (+ 905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:11:09.616671 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:11:09.616683 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:11:09.616693 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:11:09.616752 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:11:09.616768 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:11:09.616939 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0805 23:11:09.617016 (+ 77us) server_negotiation.cc:299] Negotiation successful
0805 23:11:09.617272 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":566,"thread_start_us":192,"threads_started":1}
I20260805 23:11:09.618650 3503 log.cc:824] T 00000000000000000000000000000000 P c6a7bea47f924a079fa6bfe0ad0dca69: Log is configured to *not* fsync() on all Append() calls
I20260805 23:11:09.621097 3503 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6a7bea47f924a079fa6bfe0ad0dca69: No bootstrap required, opened a new log
I20260805 23:11:09.624890 3503 raft_consensus.cc:348] T 00000000000000000000000000000000 P c6a7bea47f924a079fa6bfe0ad0dca69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6a7bea47f924a079fa6bfe0ad0dca69" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 40863 } }
I20260805 23:11:09.625197 3503 raft_consensus.cc:374] T 00000000000000000000000000000000 P c6a7bea47f924a079fa6bfe0ad0dca69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:11:09.625286 3503 raft_consensus.cc:729] T 00000000000000000000000000000000 P c6a7bea47f924a079fa6bfe0ad0dca69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6a7bea47f924a079fa6bfe0ad0dca69, State: Initialized, Role: FOLLOWER
I20260805 23:11:09.625813 3503 consensus_queue.cc:260] T 00000000000000000000000000000000 P c6a7bea47f924a079fa6bfe0ad0dca69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6a7bea47f924a079fa6bfe0ad0dca69" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 40863 } }
I20260805 23:11:09.625972 3503 raft_consensus.cc:388] T 00000000000000000000000000000000 P c6a7bea47f924a079fa6bfe0ad0dca69 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:11:09.626061 3503 raft_consensus.cc:482] T 00000000000000000000000000000000 P c6a7bea47f924a079fa6bfe0ad0dca69 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:11:09.626194 3503 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c6a7bea47f924a079fa6bfe0ad0dca69 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:11:09.627218 3503 raft_consensus.cc:504] T 00000000000000000000000000000000 P c6a7bea47f924a079fa6bfe0ad0dca69 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6a7bea47f924a079fa6bfe0ad0dca69" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 40863 } }
I20260805 23:11:09.627584 3503 leader_election.cc:302] T 00000000000000000000000000000000 P c6a7bea47f924a079fa6bfe0ad0dca69 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c6a7bea47f924a079fa6bfe0ad0dca69; no voters:
I20260805 23:11:09.627945 3503 leader_election.cc:288] T 00000000000000000000000000000000 P c6a7bea47f924a079fa6bfe0ad0dca69 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 23:11:09.628255 3508 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c6a7bea47f924a079fa6bfe0ad0dca69 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:11:09.628505 3508 raft_consensus.cc:686] T 00000000000000000000000000000000 P c6a7bea47f924a079fa6bfe0ad0dca69 [term 1 LEADER]: Becoming Leader. State: Replica: c6a7bea47f924a079fa6bfe0ad0dca69, State: Running, Role: LEADER
I20260805 23:11:09.628911 3508 consensus_queue.cc:237] T 00000000000000000000000000000000 P c6a7bea47f924a079fa6bfe0ad0dca69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6a7bea47f924a079fa6bfe0ad0dca69" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 40863 } }
I20260805 23:11:09.629292 3503 sys_catalog.cc:564] T 00000000000000000000000000000000 P c6a7bea47f924a079fa6bfe0ad0dca69 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:11:09.630703 3509 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6a7bea47f924a079fa6bfe0ad0dca69 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c6a7bea47f924a079fa6bfe0ad0dca69" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6a7bea47f924a079fa6bfe0ad0dca69" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 40863 } } }
I20260805 23:11:09.630836 3509 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6a7bea47f924a079fa6bfe0ad0dca69 [sys.catalog]: This master's current role is: LEADER
I20260805 23:11:09.631266 3517 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:11:09.631227 3510 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6a7bea47f924a079fa6bfe0ad0dca69 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c6a7bea47f924a079fa6bfe0ad0dca69. Latest consensus state: current_term: 1 leader_uuid: "c6a7bea47f924a079fa6bfe0ad0dca69" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6a7bea47f924a079fa6bfe0ad0dca69" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 40863 } } }
I20260805 23:11:09.631440 3510 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6a7bea47f924a079fa6bfe0ad0dca69 [sys.catalog]: This master's current role is: LEADER
I20260805 23:11:09.634131 3517 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:11:09.640470 3517 catalog_manager.cc:1347] Generated new cluster ID: f65a413614dc4b6b851ad7847895fefe
I20260805 23:11:09.640566 3517 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:11:09.668103 3517 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:11:09.668473 3517 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:11:09.678637 3517 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c6a7bea47f924a079fa6bfe0ad0dca69: Generated new TSK 0
I20260805 23:11:09.679508 3517 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260805 23:11:09.685583 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971406251336-25971-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971406251336-25971-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971406251336-25971-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971406251336-25971-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.92.193:0
--local_ip_for_outbound_sockets=127.25.92.193
--webserver_interface=127.25.92.193
--webserver_port=0
--tserver_master_addrs=127.25.92.254:40863
--builtin_ntp_servers=127.25.92.212:40561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260805 23:11:09.804718 3528 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:11:09.805038 3528 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:11:09.805083 3528 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:11:09.809428 3528 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:11:09.809551 3528 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:11:09.809613 3528 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.92.193
I20260805 23:11:09.814841 3528 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:40561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971406251336-25971-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971406251336-25971-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.92.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971406251336-25971-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.92.193
--webserver_port=0
--tserver_master_addrs=127.25.92.254:40863
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3528
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.92.193
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971406251336-25971-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:11:09.816028 3528 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:11:09.817497 3528 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:11:09.828074 3534 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:11:09.828079 3537 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:11:09.828074 3535 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:11:09.828351 3528 server_base.cc:1034] running on GCE node
I20260805 23:11:09.828948 3528 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:11:09.829792 3528 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:11:09.830960 3528 hybrid_clock.cc:648] HybridClock initialized: now 1785971469830897 us; error 99 us; skew 500 ppm
I20260805 23:11:09.833114 3528 webserver.cc:460] Webserver started at http://127.25.92.193:34655/ using document root <none> and password file <none>
I20260805 23:11:09.833870 3528 fs_manager.cc:359] Metadata directory not provided
I20260805 23:11:09.833945 3528 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:11:09.834178 3528 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:11:09.836447 3528 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971406251336-25971-0/minicluster-data/ts-0/data/instance:
uuid: "9c0257b833234b258196f8378bf2add1"
format_stamp: "Formatted at 2026-08-05 23:11:09 on dist-test-slave-ss4n"
server_key: "202e37d67d7379835aa04986242d7588"
server_key_iv: "68580461bd4599a218ecd142b429c296"
server_key_version: "encryptionkey@0"
I20260805 23:11:09.837117 3528 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971406251336-25971-0/minicluster-data/ts-0/wal/instance:
uuid: "9c0257b833234b258196f8378bf2add1"
format_stamp: "Formatted at 2026-08-05 23:11:09 on dist-test-slave-ss4n"
server_key: "202e37d67d7379835aa04986242d7588"
server_key_iv: "68580461bd4599a218ecd142b429c296"
server_key_version: "encryptionkey@0"
I20260805 23:11:09.841302 3528 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260805 23:11:09.844471 3543 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:11:09.846048 3528 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:11:09.846202 3528 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971406251336-25971-0/minicluster-data/ts-0/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971406251336-25971-0/minicluster-data/ts-0/wal
uuid: "9c0257b833234b258196f8378bf2add1"
format_stamp: "Formatted at 2026-08-05 23:11:09 on dist-test-slave-ss4n"
server_key: "202e37d67d7379835aa04986242d7588"
server_key_iv: "68580461bd4599a218ecd142b429c296"
server_key_version: "encryptionkey@0"
I20260805 23:11:09.846343 3528 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971406251336-25971-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971406251336-25971-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971406251336-25971-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:11:09.851259 3528 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:11:09.852074 3528 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:11:09.852303 3528 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:11:09.852898 3528 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:11:09.853921 3528 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:11:09.853998 3528 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:11:09.854069 3528 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:11:09.854120 3528 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:11:09.863565 3528 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.193:35825
I20260805 23:11:09.863673 3660 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.193:35825 every 8 connection(s)
I20260805 23:11:09.864619 3528 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971406251336-25971-0/minicluster-data/ts-0/data/info.pb
I20260805 23:11:09.871394 3506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:11:09.866897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.193:40037 (local address 127.25.92.254:40863)
0805 23:11:09.867068 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0805 23:11:09.867072 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:11:09.868189 (+ 1117us) server_negotiation.cc:407] Connection header received
0805 23:11:09.869600 (+ 1411us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:11:09.869604 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:11:09.869653 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:11:09.869735 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:11:09.870996 (+ 1261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:11:09.870998 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:11:09.871000 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:11:09.871012 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:11:09.871030 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:11:09.871038 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:11:09.871133 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0805 23:11:09.871242 (+ 109us) server_negotiation.cc:299] Negotiation successful
0805 23:11:09.871281 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260805 23:11:09.872318 3663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:11:09.867237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:40863 (local address 127.25.92.193:40037)
0805 23:11:09.867968 (+ 731us) negotiation.cc:107] Waiting for socket to connect
0805 23:11:09.868039 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0805 23:11:09.869291 (+ 1252us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:11:09.869923 (+ 632us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:11:09.869930 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:11:09.870049 (+ 119us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:11:09.870726 (+ 677us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0805 23:11:09.870728 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:11:09.870768 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0805 23:11:09.870838 (+ 70us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:11:09.871115 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:11:09.871119 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:11:09.871123 (+ 4us) client_negotiation.cc:769] Sending connection context
0805 23:11:09.871255 (+ 132us) client_negotiation.cc:240] Negotiation successful
0805 23:11:09.871454 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":123,"threads_started":1}
I20260805 23:11:09.873728 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 3528
I20260805 23:11:09.873808 3661 heartbeater.cc:347] Connected to a master server at 127.25.92.254:40863
I20260805 23:11:09.873855 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971406251336-25971-0/minicluster-data/ts-0/wal/instance
I20260805 23:11:09.874089 3661 heartbeater.cc:464] Registering TS with master...
I20260805 23:11:09.874218 25971 external_mini_cluster.cc:1428] Setting key 0a041dfc575953a9708a63ac0e075fa2
I20260805 23:11:09.874593 3661 heartbeater.cc:511] Master 127.25.92.254:40863 requested a full tablet report, sending...
I20260805 23:11:09.876166 3464 ts_manager.cc:194] Registered new tserver with Master: 9c0257b833234b258196f8378bf2add1 (127.25.92.193:35825)
I20260805 23:11:09.876272 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971406251336-25971-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971406251336-25971-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971406251336-25971-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971406251336-25971-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.92.194:0
--local_ip_for_outbound_sockets=127.25.92.194
--webserver_interface=127.25.92.194
--webserver_port=0
--tserver_master_addrs=127.25.92.254:40863
--builtin_ntp_servers=127.25.92.212:40561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260805 23:11:09.995846 3664 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:11:09.996124 3664 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:11:09.996166 3664 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:11:10.000366 3664 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:11:10.000439 3664 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:11:10.000505 3664 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.92.194
I20260805 23:11:10.005649 3664 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:40561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971406251336-25971-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971406251336-25971-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.92.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971406251336-25971-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.92.194
--webserver_port=0
--tserver_master_addrs=127.25.92.254:40863
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3664
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.92.194
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971406251336-25971-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:11:10.007117 3664 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:11:10.008661 3664 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:11:10.019161 3673 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:11:10.019161 3671 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:11:10.019266 3670 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:11:10.019526 3664 server_base.cc:1034] running on GCE node
I20260805 23:11:10.020067 3664 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:11:10.020707 3664 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:11:10.021869 3664 hybrid_clock.cc:648] HybridClock initialized: now 1785971470021839 us; error 43 us; skew 500 ppm
I20260805 23:11:10.023897 3664 webserver.cc:460] Webserver started at http://127.25.92.194:43769/ using document root <none> and password file <none>
I20260805 23:11:10.024509 3664 fs_manager.cc:359] Metadata directory not provided
I20260805 23:11:10.024576 3664 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:11:10.024811 3664 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:11:10.027050 3664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971406251336-25971-0/minicluster-data/ts-1/data/instance:
uuid: "d0376e811f834572a4201815db314b5e"
format_stamp: "Formatted at 2026-08-05 23:11:10 on dist-test-slave-ss4n"
server_key: "dd0f0a2e634b9ee61c2762df05e2bd76"
server_key_iv: "5579ae14ff057c0f4071856a66f04441"
server_key_version: "encryptionkey@0"
I20260805 23:11:10.027658 3664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971406251336-25971-0/minicluster-data/ts-1/wal/instance:
uuid: "d0376e811f834572a4201815db314b5e"
format_stamp: "Formatted at 2026-08-05 23:11:10 on dist-test-slave-ss4n"
server_key: "dd0f0a2e634b9ee61c2762df05e2bd76"
server_key_iv: "5579ae14ff057c0f4071856a66f04441"
server_key_version: "encryptionkey@0"
I20260805 23:11:10.031904 3664 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260805 23:11:10.034883 3679 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:11:10.036394 3664 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260805 23:11:10.036626 3664 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971406251336-25971-0/minicluster-data/ts-1/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971406251336-25971-0/minicluster-data/ts-1/wal
uuid: "d0376e811f834572a4201815db314b5e"
format_stamp: "Formatted at 2026-08-05 23:11:10 on dist-test-slave-ss4n"
server_key: "dd0f0a2e634b9ee61c2762df05e2bd76"
server_key_iv: "5579ae14ff057c0f4071856a66f04441"
server_key_version: "encryptionkey@0"
I20260805 23:11:10.036787 3664 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971406251336-25971-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971406251336-25971-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971406251336-25971-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:11:10.042253 3664 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:11:10.043023 3664 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:11:10.043241 3664 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:11:10.043871 3664 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:11:10.044920 3664 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:11:10.044999 3664 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:11:10.045079 3664 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:11:10.045130 3664 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:11:10.056102 3664 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.194:40033
I20260805 23:11:10.056129 3797 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.194:40033 every 8 connection(s)
I20260805 23:11:10.057156 3664 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971406251336-25971-0/minicluster-data/ts-1/data/info.pb
I20260805 23:11:10.063434 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 3664
I20260805 23:11:10.063557 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971406251336-25971-0/minicluster-data/ts-1/wal/instance
I20260805 23:11:10.063912 25971 external_mini_cluster.cc:1428] Setting key f72520044961b4cc360d48f52fc8975c
I20260805 23:11:10.064441 3506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:11:10.059325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.194:39491 (local address 127.25.92.254:40863)
0805 23:11:10.059573 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0805 23:11:10.059577 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:11:10.060680 (+ 1103us) server_negotiation.cc:407] Connection header received
0805 23:11:10.062213 (+ 1533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:11:10.062217 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:11:10.062264 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:11:10.062357 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:11:10.063881 (+ 1524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:11:10.063885 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:11:10.063888 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:11:10.063904 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:11:10.063937 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:11:10.063946 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:11:10.064068 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0805 23:11:10.064233 (+ 165us) server_negotiation.cc:299] Negotiation successful
0805 23:11:10.064290 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260805 23:11:10.066284 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971406251336-25971-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971406251336-25971-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971406251336-25971-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971406251336-25971-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.92.195:0
--local_ip_for_outbound_sockets=127.25.92.195
--webserver_interface=127.25.92.195
--webserver_port=0
--tserver_master_addrs=127.25.92.254:40863
--builtin_ntp_servers=127.25.92.212:40561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260805 23:11:10.066304 3800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:11:10.059722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:40863 (local address 127.25.92.194:39491)
0805 23:11:10.060443 (+ 721us) negotiation.cc:107] Waiting for socket to connect
0805 23:11:10.060558 (+ 115us) client_negotiation.cc:174] Beginning negotiation
0805 23:11:10.061816 (+ 1258us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:11:10.062595 (+ 779us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:11:10.062603 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:11:10.062702 (+ 99us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:11:10.063479 (+ 777us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0805 23:11:10.063481 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:11:10.063567 (+ 86us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0805 23:11:10.063696 (+ 129us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:11:10.064050 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:11:10.064060 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:11:10.064066 (+ 6us) client_negotiation.cc:769] Sending connection context
0805 23:11:10.064273 (+ 207us) client_negotiation.cc:240] Negotiation successful
0805 23:11:10.064552 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":124,"threads_started":1}
I20260805 23:11:10.067835 3798 heartbeater.cc:347] Connected to a master server at 127.25.92.254:40863
I20260805 23:11:10.068154 3798 heartbeater.cc:464] Registering TS with master...
I20260805 23:11:10.068626 3798 heartbeater.cc:511] Master 127.25.92.254:40863 requested a full tablet report, sending...
I20260805 23:11:10.069887 3464 ts_manager.cc:194] Registered new tserver with Master: d0376e811f834572a4201815db314b5e (127.25.92.194:40033)
W20260805 23:11:10.184485 3801 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:11:10.184832 3801 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:11:10.184926 3801 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:11:10.189322 3801 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:11:10.189440 3801 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:11:10.189570 3801 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.92.195
I20260805 23:11:10.194890 3801 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:40561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971406251336-25971-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971406251336-25971-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.92.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971406251336-25971-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.92.195
--webserver_port=0
--tserver_master_addrs=127.25.92.254:40863
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3801
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.92.195
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971406251336-25971-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:11:10.196133 3801 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:11:10.197640 3801 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260805 23:11:10.208149 3801 server_base.cc:1034] running on GCE node
W20260805 23:11:10.208132 3808 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:11:10.208168 3807 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:11:10.208154 3811 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:11:10.208923 3801 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:11:10.209656 3801 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:11:10.210853 3801 hybrid_clock.cc:648] HybridClock initialized: now 1785971470210833 us; error 33 us; skew 500 ppm
I20260805 23:11:10.212885 3801 webserver.cc:460] Webserver started at http://127.25.92.195:43033/ using document root <none> and password file <none>
I20260805 23:11:10.213570 3801 fs_manager.cc:359] Metadata directory not provided
I20260805 23:11:10.213672 3801 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:11:10.213891 3801 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:11:10.216168 3801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971406251336-25971-0/minicluster-data/ts-2/data/instance:
uuid: "7e11c88b4c6f4c1d9ac90ee987339e10"
format_stamp: "Formatted at 2026-08-05 23:11:10 on dist-test-slave-ss4n"
server_key: "5ef2f68b17b9cd8d8c630eb62c5583dc"
server_key_iv: "a52d39cf8baff9995d24759ad4e2c93c"
server_key_version: "encryptionkey@0"
I20260805 23:11:10.216769 3801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971406251336-25971-0/minicluster-data/ts-2/wal/instance:
uuid: "7e11c88b4c6f4c1d9ac90ee987339e10"
format_stamp: "Formatted at 2026-08-05 23:11:10 on dist-test-slave-ss4n"
server_key: "5ef2f68b17b9cd8d8c630eb62c5583dc"
server_key_iv: "a52d39cf8baff9995d24759ad4e2c93c"
server_key_version: "encryptionkey@0"
I20260805 23:11:10.220826 3801 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260805 23:11:10.223697 3817 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:11:10.225160 3801 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260805 23:11:10.225313 3801 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971406251336-25971-0/minicluster-data/ts-2/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971406251336-25971-0/minicluster-data/ts-2/wal
uuid: "7e11c88b4c6f4c1d9ac90ee987339e10"
format_stamp: "Formatted at 2026-08-05 23:11:10 on dist-test-slave-ss4n"
server_key: "5ef2f68b17b9cd8d8c630eb62c5583dc"
server_key_iv: "a52d39cf8baff9995d24759ad4e2c93c"
server_key_version: "encryptionkey@0"
I20260805 23:11:10.225446 3801 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971406251336-25971-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971406251336-25971-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971406251336-25971-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:11:10.230477 3801 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:11:10.231181 3801 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:11:10.231408 3801 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:11:10.231980 3801 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:11:10.232926 3801 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:11:10.232975 3801 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:11:10.233042 3801 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:11:10.233079 3801 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:11:10.241991 3801 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.195:33219
I20260805 23:11:10.242094 3935 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.195:33219 every 8 connection(s)
I20260805 23:11:10.243013 3801 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971406251336-25971-0/minicluster-data/ts-2/data/info.pb
I20260805 23:11:10.243642 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 3801
I20260805 23:11:10.243762 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971406251336-25971-0/minicluster-data/ts-2/wal/instance
I20260805 23:11:10.244081 25971 external_mini_cluster.cc:1428] Setting key 74d8dca13d93e7a7a649249c067fa9f6
I20260805 23:11:10.249532 3506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:11:10.245282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.195:45071 (local address 127.25.92.254:40863)
0805 23:11:10.245437 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0805 23:11:10.245442 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:11:10.246689 (+ 1247us) server_negotiation.cc:407] Connection header received
0805 23:11:10.247944 (+ 1255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:11:10.247947 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:11:10.247984 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:11:10.248055 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:11:10.249090 (+ 1035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:11:10.249093 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:11:10.249096 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:11:10.249107 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:11:10.249127 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:11:10.249135 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:11:10.249210 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0805 23:11:10.249331 (+ 121us) server_negotiation.cc:299] Negotiation successful
0805 23:11:10.249369 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260805 23:11:10.250594 3938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:11:10.245762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:40863 (local address 127.25.92.195:45071)
0805 23:11:10.246544 (+ 782us) negotiation.cc:107] Waiting for socket to connect
0805 23:11:10.246593 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0805 23:11:10.247757 (+ 1164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:11:10.248215 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:11:10.248221 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:11:10.248306 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:11:10.248847 (+ 541us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0805 23:11:10.248849 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:11:10.248889 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0805 23:11:10.248956 (+ 67us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:11:10.249212 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:11:10.249218 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:11:10.249222 (+ 4us) client_negotiation.cc:769] Sending connection context
0805 23:11:10.249351 (+ 129us) client_negotiation.cc:240] Negotiation successful
0805 23:11:10.249718 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":120,"threads_started":1}
I20260805 23:11:10.251763 3936 heartbeater.cc:347] Connected to a master server at 127.25.92.254:40863
I20260805 23:11:10.251966 3936 heartbeater.cc:464] Registering TS with master...
I20260805 23:11:10.252354 3936 heartbeater.cc:511] Master 127.25.92.254:40863 requested a full tablet report, sending...
I20260805 23:11:10.253340 3464 ts_manager.cc:194] Registered new tserver with Master: 7e11c88b4c6f4c1d9ac90ee987339e10 (127.25.92.195:33219)
I20260805 23:11:10.258378 25971 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260805 23:11:10.261385 3506 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:11:10.260549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38246 (local address 127.25.92.254:40863)
0805 23:11:10.260684 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0805 23:11:10.260688 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:11:10.260821 (+ 133us) server_negotiation.cc:407] Connection header received
0805 23:11:10.260960 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:11:10.260963 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:11:10.261000 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:11:10.261069 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:11:10.261270 (+ 201us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38246: BlockingRecv error: recv got EOF from 127.0.0.1:38246 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
I20260805 23:11:10.261924 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 3528
I20260805 23:11:10.269073 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 3664
I20260805 23:11:10.274884 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 3801
I20260805 23:11:10.282153 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 3434
2026-08-05T23:11:10Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (889 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971406251336-25971-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 23:11:10 dist-test-slave-ss4n krb5kdc[3946](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 23:11:10 dist-test-slave-ss4n krb5kdc[3946](info): set up 2 sockets
Aug 05 23:11:10 dist-test-slave-ss4n krb5kdc[3946](info): commencing operation
krb5kdc: starting...
W20260805 23:11:11.389452 25971 mini_kdc.cc:121] Time spent starting KDC: real 1.086s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971406251336-25971-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971406251336-25971-0/krb5kdc/test-user.keytab.
Aug 05 23:11:11 dist-test-slave-ss4n krb5kdc[3946](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971471, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T23:11:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:11:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.92.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.92.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.92.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab.
Entry for principal kudu/127.25.92.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab.
I20260805 23:11:11.534200 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971406251336-25971-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971406251336-25971-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971406251336-25971-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.254:35843
--webserver_interface=127.25.92.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab
--principal=kudu/127.25.92.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:34815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.92.254:35843
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:11:11.656939 3959 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:11:11.657263 3959 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:11:11.657308 3959 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:11:11.661880 3959 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:11:11.661958 3959 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:11:11.661979 3959 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:11:11.661998 3959 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:11:11.662014 3959 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:11:11.667959 3959 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:34815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971406251336-25971-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971406251336-25971-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.92.254:35843
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab
--principal=kudu/127.25.92.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.254:35843
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.92.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3959
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971406251336-25971-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:11:11.669220 3959 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:11:11.670346 3959 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:11:11.679901 3966 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:11:11.679896 3965 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:11:11.679896 3968 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:11:11.680305 3959 server_base.cc:1034] running on GCE node
I20260805 23:11:11.681052 3959 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:11:11.682350 3959 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:11:11.683641 3959 hybrid_clock.cc:648] HybridClock initialized: now 1785971471683624 us; error 35 us; skew 500 ppm
Aug 05 23:11:11 dist-test-slave-ss4n krb5kdc[3946](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971471, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:11:11.687000 3959 init.cc:377] Logged in from keytab as kudu/127.25.92.254@KRBTEST.COM (short username kudu)
I20260805 23:11:11.688088 3959 webserver.cc:460] Webserver started at http://127.25.92.254:46249/ using document root <none> and password file <none>
I20260805 23:11:11.688740 3959 fs_manager.cc:359] Metadata directory not provided
I20260805 23:11:11.688838 3959 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:11:11.689088 3959 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:11:11.691478 3959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971406251336-25971-0/minicluster-data/master-0/data/instance:
uuid: "f2fd73772ea04ffa8368c49ece99103a"
format_stamp: "Formatted at 2026-08-05 23:11:11 on dist-test-slave-ss4n"
server_key: "4a01b49c9cadac18cbe841559384a078"
server_key_iv: "1110e736d98971afe60eca34212ec81d"
server_key_version: "encryptionkey@0"
I20260805 23:11:11.692121 3959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971406251336-25971-0/minicluster-data/master-0/wal/instance:
uuid: "f2fd73772ea04ffa8368c49ece99103a"
format_stamp: "Formatted at 2026-08-05 23:11:11 on dist-test-slave-ss4n"
server_key: "4a01b49c9cadac18cbe841559384a078"
server_key_iv: "1110e736d98971afe60eca34212ec81d"
server_key_version: "encryptionkey@0"
I20260805 23:11:11.696391 3959 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260805 23:11:11.699328 3975 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:11:11.700685 3959 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260805 23:11:11.700851 3959 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971406251336-25971-0/minicluster-data/master-0/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971406251336-25971-0/minicluster-data/master-0/wal
uuid: "f2fd73772ea04ffa8368c49ece99103a"
format_stamp: "Formatted at 2026-08-05 23:11:11 on dist-test-slave-ss4n"
server_key: "4a01b49c9cadac18cbe841559384a078"
server_key_iv: "1110e736d98971afe60eca34212ec81d"
server_key_version: "encryptionkey@0"
I20260805 23:11:11.700995 3959 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971406251336-25971-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971406251336-25971-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971406251336-25971-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:11:11.719072 3959 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:11:11.722970 3959 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:11:11.723228 3959 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:11:11.731808 3959 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.254:35843
I20260805 23:11:11.731881 4028 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.254:35843 every 8 connection(s)
I20260805 23:11:11.732951 3959 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
I20260805 23:11:11.736111 4029 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:11:11.742352 4029 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2fd73772ea04ffa8368c49ece99103a: Bootstrap starting.
I20260805 23:11:11.742717 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 3959
I20260805 23:11:11.742847 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971406251336-25971-0/minicluster-data/master-0/wal/instance
I20260805 23:11:11.743145 25971 external_mini_cluster.cc:1428] Setting key 602b9eb6b6878632e1c26b7fb9ae8a52
I20260805 23:11:11.745469 4029 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f2fd73772ea04ffa8368c49ece99103a: Neither blocks nor log segments found. Creating new log.
I20260805 23:11:11.746362 4029 log.cc:824] T 00000000000000000000000000000000 P f2fd73772ea04ffa8368c49ece99103a: Log is configured to *not* fsync() on all Append() calls
I20260805 23:11:11.748725 4029 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2fd73772ea04ffa8368c49ece99103a: No bootstrap required, opened a new log
Aug 05 23:11:11 dist-test-slave-ss4n krb5kdc[3946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971471, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM
I20260805 23:11:11.754351 4029 raft_consensus.cc:348] T 00000000000000000000000000000000 P f2fd73772ea04ffa8368c49ece99103a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2fd73772ea04ffa8368c49ece99103a" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 35843 } }
I20260805 23:11:11.754746 4029 raft_consensus.cc:374] T 00000000000000000000000000000000 P f2fd73772ea04ffa8368c49ece99103a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:11:11.754815 4029 raft_consensus.cc:729] T 00000000000000000000000000000000 P f2fd73772ea04ffa8368c49ece99103a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2fd73772ea04ffa8368c49ece99103a, State: Initialized, Role: FOLLOWER
I20260805 23:11:11.755358 4029 consensus_queue.cc:260] T 00000000000000000000000000000000 P f2fd73772ea04ffa8368c49ece99103a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2fd73772ea04ffa8368c49ece99103a" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 35843 } }
I20260805 23:11:11.755520 4029 raft_consensus.cc:388] T 00000000000000000000000000000000 P f2fd73772ea04ffa8368c49ece99103a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:11:11.755654 4029 raft_consensus.cc:482] T 00000000000000000000000000000000 P f2fd73772ea04ffa8368c49ece99103a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:11:11.755780 4029 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f2fd73772ea04ffa8368c49ece99103a [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:11:11.757205 4029 raft_consensus.cc:504] T 00000000000000000000000000000000 P f2fd73772ea04ffa8368c49ece99103a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2fd73772ea04ffa8368c49ece99103a" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 35843 } }
I20260805 23:11:11.757673 4029 leader_election.cc:302] T 00000000000000000000000000000000 P f2fd73772ea04ffa8368c49ece99103a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f2fd73772ea04ffa8368c49ece99103a; no voters:
I20260805 23:11:11.758119 4029 leader_election.cc:288] T 00000000000000000000000000000000 P f2fd73772ea04ffa8368c49ece99103a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 23:11:11.758236 4035 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f2fd73772ea04ffa8368c49ece99103a [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:11:11.758466 4035 raft_consensus.cc:686] T 00000000000000000000000000000000 P f2fd73772ea04ffa8368c49ece99103a [term 1 LEADER]: Becoming Leader. State: Replica: f2fd73772ea04ffa8368c49ece99103a, State: Running, Role: LEADER
I20260805 23:11:11.758919 4035 consensus_queue.cc:237] T 00000000000000000000000000000000 P f2fd73772ea04ffa8368c49ece99103a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2fd73772ea04ffa8368c49ece99103a" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 35843 } }
I20260805 23:11:11.759657 4029 sys_catalog.cc:564] T 00000000000000000000000000000000 P f2fd73772ea04ffa8368c49ece99103a [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:11:11.761111 4036 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2fd73772ea04ffa8368c49ece99103a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f2fd73772ea04ffa8368c49ece99103a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2fd73772ea04ffa8368c49ece99103a" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 35843 } } }
I20260805 23:11:11.761101 4037 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2fd73772ea04ffa8368c49ece99103a [sys.catalog]: SysCatalogTable state changed. Reason: New leader f2fd73772ea04ffa8368c49ece99103a. Latest consensus state: current_term: 1 leader_uuid: "f2fd73772ea04ffa8368c49ece99103a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2fd73772ea04ffa8368c49ece99103a" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 35843 } } }
I20260805 23:11:11.761230 4032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:11:11.744702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45388 (local address 127.25.92.254:35843)
0805 23:11:11.745252 (+ 550us) server_negotiation.cc:206] Beginning negotiation
0805 23:11:11.745268 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0805 23:11:11.745306 (+ 38us) server_negotiation.cc:407] Connection header received
0805 23:11:11.746159 (+ 853us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:11:11.746180 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:11:11.746593 (+ 413us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:11:11.746980 (+ 387us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:11:11.747906 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:11:11.749260 (+ 1354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:11:11.750721 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:11:11.751248 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:11:11.751388 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:11:11.754977 (+ 3589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:11:11.754983 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:11:11.754999 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:11:11.755058 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:11:11.757313 (+ 2255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:11:11.758004 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:11:11.758012 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:11:11.758021 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:11:11.758200 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:11:11.758569 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:11:11.758572 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:11:11.758574 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:11:11.759116 (+ 542us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:11:11.759332 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0805 23:11:11.759646 (+ 314us) server_negotiation.cc:299] Negotiation successful
0805 23:11:11.759986 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":165,"threads_started":1}
I20260805 23:11:11.761264 4036 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2fd73772ea04ffa8368c49ece99103a [sys.catalog]: This master's current role is: LEADER
I20260805 23:11:11.761264 4037 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2fd73772ea04ffa8368c49ece99103a [sys.catalog]: This master's current role is: LEADER
I20260805 23:11:11.762295 4047 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:11:11.765873 4047 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:11:11.772090 4047 catalog_manager.cc:1347] Generated new cluster ID: 5bf9d8902d3f4c6fbd5ffc4e3d102ce6
I20260805 23:11:11.772190 4047 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:11:11.782681 4047 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:11:11.783905 4047 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:11:11.791404 4047 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f2fd73772ea04ffa8368c49ece99103a: Generated new TSK 0
I20260805 23:11:11.792367 4047 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.92.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.92.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.92.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.193.keytab.
Entry for principal kudu/127.25.92.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.193.keytab.
I20260805 23:11:11.829949 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971406251336-25971-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971406251336-25971-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971406251336-25971-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971406251336-25971-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.92.193:0
--local_ip_for_outbound_sockets=127.25.92.193
--webserver_interface=127.25.92.193
--webserver_port=0
--tserver_master_addrs=127.25.92.254:35843
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.193.keytab
--principal=kudu/127.25.92.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:34815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:11:11.951516 4059 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:11:11.951872 4059 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:11:11.951956 4059 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:11:11.956204 4059 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:11:11.956318 4059 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:11:11.956444 4059 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.92.193
I20260805 23:11:11.962363 4059 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:34815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971406251336-25971-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971406251336-25971-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.193.keytab
--principal=kudu/127.25.92.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.92.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971406251336-25971-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.92.193
--webserver_port=0
--tserver_master_addrs=127.25.92.254:35843
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4059
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.92.193
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971406251336-25971-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:11:11.963595 4059 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:11:11.964607 4059 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260805 23:11:11.974891 4059 server_base.cc:1034] running on GCE node
W20260805 23:11:11.974833 4065 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:11:11.974826 4066 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:11:11.975263 4068 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:11:11.975683 4059 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:11:11.976354 4059 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:11:11.977586 4059 hybrid_clock.cc:648] HybridClock initialized: now 1785971471977560 us; error 49 us; skew 500 ppm
Aug 05 23:11:11 dist-test-slave-ss4n krb5kdc[3946](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971471, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:11:11.980509 4059 init.cc:377] Logged in from keytab as kudu/127.25.92.193@KRBTEST.COM (short username kudu)
I20260805 23:11:11.981377 4059 webserver.cc:460] Webserver started at http://127.25.92.193:46613/ using document root <none> and password file <none>
I20260805 23:11:11.981972 4059 fs_manager.cc:359] Metadata directory not provided
I20260805 23:11:11.982034 4059 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:11:11.982226 4059 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:11:11.984227 4059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971406251336-25971-0/minicluster-data/ts-0/data/instance:
uuid: "d889302783554142a0a7d93e65088548"
format_stamp: "Formatted at 2026-08-05 23:11:11 on dist-test-slave-ss4n"
server_key: "7d8bca16ba991470d8fd8f4575234fb6"
server_key_iv: "c09dc2adff1f040024e2fda3f607f6dd"
server_key_version: "encryptionkey@0"
I20260805 23:11:11.984745 4059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971406251336-25971-0/minicluster-data/ts-0/wal/instance:
uuid: "d889302783554142a0a7d93e65088548"
format_stamp: "Formatted at 2026-08-05 23:11:11 on dist-test-slave-ss4n"
server_key: "7d8bca16ba991470d8fd8f4575234fb6"
server_key_iv: "c09dc2adff1f040024e2fda3f607f6dd"
server_key_version: "encryptionkey@0"
I20260805 23:11:11.988579 4059 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260805 23:11:11.991338 4076 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:11:11.992633 4059 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260805 23:11:11.992766 4059 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971406251336-25971-0/minicluster-data/ts-0/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971406251336-25971-0/minicluster-data/ts-0/wal
uuid: "d889302783554142a0a7d93e65088548"
format_stamp: "Formatted at 2026-08-05 23:11:11 on dist-test-slave-ss4n"
server_key: "7d8bca16ba991470d8fd8f4575234fb6"
server_key_iv: "c09dc2adff1f040024e2fda3f607f6dd"
server_key_version: "encryptionkey@0"
I20260805 23:11:11.992893 4059 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971406251336-25971-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971406251336-25971-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971406251336-25971-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:11:12.009799 4059 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:11:12.013243 4059 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:11:12.013410 4059 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:11:12.014060 4059 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:11:12.014928 4059 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:11:12.015009 4059 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:11:12.015072 4059 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:11:12.015128 4059 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:11:12.024585 4059 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.193:39815
I20260805 23:11:12.024603 4202 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.193:39815 every 8 connection(s)
I20260805 23:11:12.025555 4059 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971406251336-25971-0/minicluster-data/ts-0/data/info.pb
I20260805 23:11:12.027928 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 4059
I20260805 23:11:12.028086 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971406251336-25971-0/minicluster-data/ts-0/wal/instance
I20260805 23:11:12.028466 25971 external_mini_cluster.cc:1428] Setting key 57a1e03c90b33e5af2d7a56f5f09659c
Aug 05 23:11:12 dist-test-slave-ss4n krb5kdc[3946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971471, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.193@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.25.92.194@KRBTEST.COM; defaulting to no policy
I20260805 23:11:12.042306 4032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:11:12.027540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.193:57795 (local address 127.25.92.254:35843)
0805 23:11:12.027739 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0805 23:11:12.027745 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:11:12.028734 (+ 989us) server_negotiation.cc:407] Connection header received
0805 23:11:12.030630 (+ 1896us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:11:12.030633 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:11:12.030679 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:11:12.030757 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:11:12.032449 (+ 1692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:11:12.033006 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:11:12.034274 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:11:12.034686 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:11:12.034814 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:11:12.037939 (+ 3125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:11:12.037946 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:11:12.037949 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:11:12.037985 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:11:12.039877 (+ 1892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:11:12.040545 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:11:12.040549 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:11:12.040551 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:11:12.040658 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:11:12.041063 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:11:12.041066 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:11:12.041067 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:11:12.041274 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:11:12.041352 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0805 23:11:12.042023 (+ 671us) server_negotiation.cc:299] Negotiation successful
0805 23:11:12.042153 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260805 23:11:12.043206 4206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:11:12.027928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:35843 (local address 127.25.92.193:57795)
0805 23:11:12.028530 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0805 23:11:12.028607 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0805 23:11:12.030337 (+ 1730us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:11:12.030985 (+ 648us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:11:12.030997 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:11:12.031556 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:11:12.032293 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:11:12.032309 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:11:12.033150 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:11:12.033154 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:11:12.034134 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:11:12.034144 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:11:12.034872 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:11:12.034875 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:11:12.035072 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:11:12.035809 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:11:12.035826 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:11:12.037730 (+ 1904us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:11:12.040023 (+ 2293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:11:12.040029 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:11:12.040045 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:11:12.040369 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:11:12.040787 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:11:12.040790 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:11:12.040791 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:11:12.040957 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:11:12.041380 (+ 423us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:11:12.041386 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:11:12.041742 (+ 356us) client_negotiation.cc:769] Sending connection context
0805 23:11:12.041912 (+ 170us) client_negotiation.cc:240] Negotiation successful
0805 23:11:12.042258 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":183,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.92.194@KRBTEST.COM" created.
I20260805 23:11:12.044703 4204 heartbeater.cc:347] Connected to a master server at 127.25.92.254:35843
I20260805 23:11:12.044974 4204 heartbeater.cc:464] Registering TS with master...
I20260805 23:11:12.045639 4204 heartbeater.cc:511] Master 127.25.92.254:35843 requested a full tablet report, sending...
I20260805 23:11:12.047416 3991 ts_manager.cc:194] Registered new tserver with Master: d889302783554142a0a7d93e65088548 (127.25.92.193:39815)
I20260805 23:11:12.049142 3991 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.92.193@KRBTEST.COM'} at 127.25.92.193:57795
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.92.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.194.keytab.
Entry for principal kudu/127.25.92.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.194.keytab.
I20260805 23:11:12.061192 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971406251336-25971-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971406251336-25971-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971406251336-25971-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971406251336-25971-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.92.194:0
--local_ip_for_outbound_sockets=127.25.92.194
--webserver_interface=127.25.92.194
--webserver_port=0
--tserver_master_addrs=127.25.92.254:35843
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.194.keytab
--principal=kudu/127.25.92.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:34815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:11:12.175973 4209 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:11:12.176261 4209 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:11:12.176307 4209 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:11:12.180395 4209 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:11:12.180480 4209 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:11:12.180544 4209 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.92.194
I20260805 23:11:12.186067 4209 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:34815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971406251336-25971-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971406251336-25971-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.194.keytab
--principal=kudu/127.25.92.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.92.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971406251336-25971-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.92.194
--webserver_port=0
--tserver_master_addrs=127.25.92.254:35843
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4209
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.92.194
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971406251336-25971-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:11:12.187193 4209 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:11:12.188269 4209 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:11:12.198812 4215 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:11:12.198904 4209 server_base.cc:1034] running on GCE node
W20260805 23:11:12.198913 4216 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:11:12.198830 4219 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:11:12.199532 4209 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:11:12.200208 4209 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:11:12.201432 4209 hybrid_clock.cc:648] HybridClock initialized: now 1785971472201396 us; error 45 us; skew 500 ppm
Aug 05 23:11:12 dist-test-slave-ss4n krb5kdc[3946](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971472, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:11:12.205170 4209 init.cc:377] Logged in from keytab as kudu/127.25.92.194@KRBTEST.COM (short username kudu)
I20260805 23:11:12.206321 4209 webserver.cc:460] Webserver started at http://127.25.92.194:35521/ using document root <none> and password file <none>
I20260805 23:11:12.206940 4209 fs_manager.cc:359] Metadata directory not provided
I20260805 23:11:12.207043 4209 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:11:12.207286 4209 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:11:12.209581 4209 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971406251336-25971-0/minicluster-data/ts-1/data/instance:
uuid: "0405e56fcd704035b34b2ac37cad4597"
format_stamp: "Formatted at 2026-08-05 23:11:12 on dist-test-slave-ss4n"
server_key: "ffc16606a7a854d3a7910169f9f199e8"
server_key_iv: "7e8d6d55ede78d61d7013effcda30d5e"
server_key_version: "encryptionkey@0"
I20260805 23:11:12.210191 4209 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971406251336-25971-0/minicluster-data/ts-1/wal/instance:
uuid: "0405e56fcd704035b34b2ac37cad4597"
format_stamp: "Formatted at 2026-08-05 23:11:12 on dist-test-slave-ss4n"
server_key: "ffc16606a7a854d3a7910169f9f199e8"
server_key_iv: "7e8d6d55ede78d61d7013effcda30d5e"
server_key_version: "encryptionkey@0"
I20260805 23:11:12.214267 4209 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260805 23:11:12.216938 4226 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:11:12.218389 4209 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260805 23:11:12.218629 4209 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971406251336-25971-0/minicluster-data/ts-1/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971406251336-25971-0/minicluster-data/ts-1/wal
uuid: "0405e56fcd704035b34b2ac37cad4597"
format_stamp: "Formatted at 2026-08-05 23:11:12 on dist-test-slave-ss4n"
server_key: "ffc16606a7a854d3a7910169f9f199e8"
server_key_iv: "7e8d6d55ede78d61d7013effcda30d5e"
server_key_version: "encryptionkey@0"
I20260805 23:11:12.218781 4209 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971406251336-25971-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971406251336-25971-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971406251336-25971-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:11:12.233412 4209 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:11:12.237026 4209 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:11:12.237248 4209 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:11:12.237936 4209 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:11:12.238818 4209 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:11:12.238896 4209 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:11:12.238966 4209 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:11:12.239022 4209 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:11:12.247998 4209 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.194:41563
I20260805 23:11:12.248047 4346 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.194:41563 every 8 connection(s)
I20260805 23:11:12.248960 4209 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971406251336-25971-0/minicluster-data/ts-1/data/info.pb
I20260805 23:11:12.258639 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 4209
I20260805 23:11:12.258745 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971406251336-25971-0/minicluster-data/ts-1/wal/instance
Aug 05 23:11:12 dist-test-slave-ss4n krb5kdc[3946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971472, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.194@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM
I20260805 23:11:12.259071 25971 external_mini_cluster.cc:1428] Setting key d5eb4c2c8d827ef98dbb2b43d3dbb3c2
I20260805 23:11:12.264597 4032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:11:12.250945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.194:44133 (local address 127.25.92.254:35843)
0805 23:11:12.251127 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0805 23:11:12.251131 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:11:12.251944 (+ 813us) server_negotiation.cc:407] Connection header received
0805 23:11:12.253405 (+ 1461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:11:12.253409 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:11:12.253528 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:11:12.253631 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:11:12.254807 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:11:12.255351 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:11:12.256536 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:11:12.256762 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:11:12.256876 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:11:12.259732 (+ 2856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:11:12.259736 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:11:12.259739 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:11:12.259772 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:11:12.261886 (+ 2114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:11:12.262633 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:11:12.262636 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:11:12.262638 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:11:12.262723 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:11:12.263233 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:11:12.263236 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:11:12.263238 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:11:12.263438 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:11:12.263533 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0805 23:11:12.264270 (+ 737us) server_negotiation.cc:299] Negotiation successful
0805 23:11:12.264413 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260805 23:11:12.265709 4350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:11:12.251331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:35843 (local address 127.25.92.194:44133)
0805 23:11:12.251798 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0805 23:11:12.251839 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0805 23:11:12.253188 (+ 1349us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:11:12.253808 (+ 620us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:11:12.253815 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:11:12.254179 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:11:12.254663 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:11:12.254676 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:11:12.255491 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:11:12.255496 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:11:12.256373 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:11:12.256382 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:11:12.256917 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:11:12.256921 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:11:12.257093 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:11:12.257790 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:11:12.257807 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:11:12.259462 (+ 1655us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:11:12.262050 (+ 2588us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:11:12.262057 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:11:12.262074 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:11:12.262516 (+ 442us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:11:12.262829 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:11:12.262834 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:11:12.262839 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:11:12.263070 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:11:12.263570 (+ 500us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:11:12.263582 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:11:12.263959 (+ 377us) client_negotiation.cc:769] Sending connection context
0805 23:11:12.264225 (+ 266us) client_negotiation.cc:240] Negotiation successful
0805 23:11:12.264617 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":119,"threads_started":1}
I20260805 23:11:12.267062 4348 heartbeater.cc:347] Connected to a master server at 127.25.92.254:35843
I20260805 23:11:12.267352 4348 heartbeater.cc:464] Registering TS with master...
I20260805 23:11:12.267876 4348 heartbeater.cc:511] Master 127.25.92.254:35843 requested a full tablet report, sending...
I20260805 23:11:12.269065 3991 ts_manager.cc:194] Registered new tserver with Master: 0405e56fcd704035b34b2ac37cad4597 (127.25.92.194:41563)
WARNING: no policy specified for kudu/127.25.92.195@KRBTEST.COM; defaulting to no policy
I20260805 23:11:12.269943 3991 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.92.194@KRBTEST.COM'} at 127.25.92.194:44133
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.92.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.92.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.195.keytab.
Entry for principal kudu/127.25.92.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.195.keytab.
I20260805 23:11:12.289337 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971406251336-25971-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971406251336-25971-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971406251336-25971-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971406251336-25971-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.92.195:0
--local_ip_for_outbound_sockets=127.25.92.195
--webserver_interface=127.25.92.195
--webserver_port=0
--tserver_master_addrs=127.25.92.254:35843
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.195.keytab
--principal=kudu/127.25.92.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:34815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:11:12.414004 4353 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:11:12.414310 4353 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:11:12.414356 4353 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:11:12.418519 4353 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:11:12.418606 4353 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:11:12.418669 4353 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.92.195
I20260805 23:11:12.424551 4353 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:34815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971406251336-25971-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971406251336-25971-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.195.keytab
--principal=kudu/127.25.92.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.92.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971406251336-25971-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.92.195
--webserver_port=0
--tserver_master_addrs=127.25.92.254:35843
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4353
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.92.195
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971406251336-25971-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:11:12.425701 4353 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:11:12.426733 4353 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260805 23:11:12.437377 4353 server_base.cc:1034] running on GCE node
W20260805 23:11:12.437321 4358 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:11:12.437321 4359 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:11:12.437315 4361 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:11:12.438208 4353 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:11:12.439019 4353 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:11:12.440270 4353 hybrid_clock.cc:648] HybridClock initialized: now 1785971472440233 us; error 54 us; skew 500 ppm
Aug 05 23:11:12 dist-test-slave-ss4n krb5kdc[3946](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971472, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:11:12.443892 4353 init.cc:377] Logged in from keytab as kudu/127.25.92.195@KRBTEST.COM (short username kudu)
I20260805 23:11:12.444958 4353 webserver.cc:460] Webserver started at http://127.25.92.195:46811/ using document root <none> and password file <none>
I20260805 23:11:12.445647 4353 fs_manager.cc:359] Metadata directory not provided
I20260805 23:11:12.445751 4353 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:11:12.446002 4353 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:11:12.448324 4353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971406251336-25971-0/minicluster-data/ts-2/data/instance:
uuid: "d8b743ce4f814feba02e7b48af42f02a"
format_stamp: "Formatted at 2026-08-05 23:11:12 on dist-test-slave-ss4n"
server_key: "b3fc108e96e827673cfc2302abbb556c"
server_key_iv: "dfc18716f0447689a01e0ac1a774df67"
server_key_version: "encryptionkey@0"
I20260805 23:11:12.448978 4353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971406251336-25971-0/minicluster-data/ts-2/wal/instance:
uuid: "d8b743ce4f814feba02e7b48af42f02a"
format_stamp: "Formatted at 2026-08-05 23:11:12 on dist-test-slave-ss4n"
server_key: "b3fc108e96e827673cfc2302abbb556c"
server_key_iv: "dfc18716f0447689a01e0ac1a774df67"
server_key_version: "encryptionkey@0"
I20260805 23:11:12.453135 4353 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260805 23:11:12.455901 4368 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:11:12.457412 4353 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260805 23:11:12.457619 4353 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971406251336-25971-0/minicluster-data/ts-2/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971406251336-25971-0/minicluster-data/ts-2/wal
uuid: "d8b743ce4f814feba02e7b48af42f02a"
format_stamp: "Formatted at 2026-08-05 23:11:12 on dist-test-slave-ss4n"
server_key: "b3fc108e96e827673cfc2302abbb556c"
server_key_iv: "dfc18716f0447689a01e0ac1a774df67"
server_key_version: "encryptionkey@0"
I20260805 23:11:12.457763 4353 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971406251336-25971-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971406251336-25971-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971406251336-25971-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:11:12.479789 4353 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:11:12.483412 4353 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:11:12.483645 4353 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:11:12.484275 4353 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:11:12.485153 4353 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:11:12.485226 4353 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:11:12.485301 4353 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:11:12.485343 4353 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:11:12.495018 4353 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.195:41317
I20260805 23:11:12.495035 4487 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.195:41317 every 8 connection(s)
I20260805 23:11:12.496080 4353 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971406251336-25971-0/minicluster-data/ts-2/data/info.pb
I20260805 23:11:12.497444 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 4353
I20260805 23:11:12.497630 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971406251336-25971-0/minicluster-data/ts-2/wal/instance
I20260805 23:11:12.497910 25971 external_mini_cluster.cc:1428] Setting key 99d63aa4bcc20d4d16d6092881917f46
Aug 05 23:11:12 dist-test-slave-ss4n krb5kdc[3946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971472, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.195@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM
I20260805 23:11:12.512449 4032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:11:12.498388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.195:58589 (local address 127.25.92.254:35843)
0805 23:11:12.498569 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0805 23:11:12.498574 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:11:12.499613 (+ 1039us) server_negotiation.cc:407] Connection header received
0805 23:11:12.501219 (+ 1606us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:11:12.501222 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:11:12.501269 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:11:12.501339 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:11:12.502735 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:11:12.503304 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:11:12.504561 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:11:12.504819 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:11:12.504910 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:11:12.507875 (+ 2965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:11:12.507878 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:11:12.507880 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:11:12.507910 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:11:12.509569 (+ 1659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:11:12.510514 (+ 945us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:11:12.510518 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:11:12.510519 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:11:12.510639 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:11:12.511137 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:11:12.511140 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:11:12.511142 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:11:12.511374 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:11:12.511504 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0805 23:11:12.512192 (+ 688us) server_negotiation.cc:299] Negotiation successful
0805 23:11:12.512323 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260805 23:11:12.513391 4490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:11:12.498876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:35843 (local address 127.25.92.195:58589)
0805 23:11:12.499428 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0805 23:11:12.499498 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0805 23:11:12.501001 (+ 1503us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:11:12.501561 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:11:12.501571 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:11:12.502010 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:11:12.502570 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:11:12.502585 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:11:12.503454 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:11:12.503457 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:11:12.504421 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:11:12.504431 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:11:12.504968 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:11:12.504971 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:11:12.505132 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:11:12.505911 (+ 779us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:11:12.505930 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:11:12.507637 (+ 1707us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:11:12.509744 (+ 2107us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:11:12.509751 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:11:12.509776 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:11:12.510078 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:11:12.510752 (+ 674us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:11:12.510756 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:11:12.510758 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:11:12.510994 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:11:12.511515 (+ 521us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:11:12.511521 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:11:12.511901 (+ 380us) client_negotiation.cc:769] Sending connection context
0805 23:11:12.512134 (+ 233us) client_negotiation.cc:240] Negotiation successful
0805 23:11:12.512445 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":160,"threads_started":1}
I20260805 23:11:12.514698 4488 heartbeater.cc:347] Connected to a master server at 127.25.92.254:35843
I20260805 23:11:12.514976 4488 heartbeater.cc:464] Registering TS with master...
I20260805 23:11:12.515508 4488 heartbeater.cc:511] Master 127.25.92.254:35843 requested a full tablet report, sending...
I20260805 23:11:12.516577 3991 ts_manager.cc:194] Registered new tserver with Master: d8b743ce4f814feba02e7b48af42f02a (127.25.92.195:41317)
I20260805 23:11:12.517176 3991 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.92.195@KRBTEST.COM'} at 127.25.92.195:58589
I20260805 23:11:12.524365 25971 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260805 23:11:12.533936 4032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:11:12.526460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47640 (local address 127.25.92.254:35843)
0805 23:11:12.526718 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0805 23:11:12.526723 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:11:12.526743 (+ 20us) server_negotiation.cc:407] Connection header received
0805 23:11:12.526801 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:11:12.526804 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:11:12.526848 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:11:12.526928 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:11:12.527643 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:11:12.528212 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:11:12.529021 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:11:12.529273 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:11:12.529366 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:11:12.531020 (+ 1654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:11:12.531023 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:11:12.531025 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:11:12.531053 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:11:12.532327 (+ 1274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:11:12.532727 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:11:12.532730 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:11:12.532731 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:11:12.532805 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:11:12.533042 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:11:12.533044 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:11:12.533045 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:11:12.533271 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:11:12.533386 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0805 23:11:12.533691 (+ 305us) server_negotiation.cc:299] Negotiation successful
0805 23:11:12.533811 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162}
I20260805 23:11:12.536926 3991 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47640:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260805 23:11:12.540130 3991 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260805 23:11:12.555846 4504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:11:12.550582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43168 (local address 127.25.92.194:41563)
0805 23:11:12.551291 (+ 709us) server_negotiation.cc:206] Beginning negotiation
0805 23:11:12.551299 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0805 23:11:12.551315 (+ 16us) server_negotiation.cc:407] Connection header received
0805 23:11:12.551371 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:11:12.551374 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:11:12.551501 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:11:12.551650 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:11:12.551943 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:11:12.552657 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:11:12.554366 (+ 1709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:11:12.555126 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:11:12.555255 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:11:12.555407 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0805 23:11:12.555617 (+ 210us) server_negotiation.cc:299] Negotiation successful
0805 23:11:12.555712 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":629,"thread_start_us":143,"threads_started":1}
I20260805 23:11:12.555820 4503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:11:12.550182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.194:41563 (local address 127.0.0.1:43168)
0805 23:11:12.550984 (+ 802us) negotiation.cc:107] Waiting for socket to connect
0805 23:11:12.550996 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0805 23:11:12.551067 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:11:12.551641 (+ 574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:11:12.551644 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:11:12.551670 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:11:12.551797 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:11:12.551806 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:11:12.552792 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:11:12.552799 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:11:12.554241 (+ 1442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:11:12.554251 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:11:12.555270 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:11:12.555273 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:11:12.555362 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:11:12.555393 (+ 31us) client_negotiation.cc:769] Sending connection context
0805 23:11:12.555526 (+ 133us) client_negotiation.cc:240] Negotiation successful
0805 23:11:12.555659 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":738,"mutex_wait_us":207,"thread_start_us":90,"threads_started":1}
I20260805 23:11:12.556265 4500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:11:12.549530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56290 (local address 127.25.92.193:39815)
0805 23:11:12.549990 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0805 23:11:12.549998 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0805 23:11:12.550295 (+ 297us) server_negotiation.cc:407] Connection header received
0805 23:11:12.550607 (+ 312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:11:12.550616 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:11:12.550829 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:11:12.550954 (+ 125us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:11:12.551666 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:11:12.552861 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:11:12.554664 (+ 1803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:11:12.555443 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:11:12.555547 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:11:12.555693 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0805 23:11:12.556017 (+ 324us) server_negotiation.cc:299] Negotiation successful
0805 23:11:12.556117 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":147,"threads_started":1}
I20260805 23:11:12.556643 4499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:11:12.549342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.193:39815 (local address 127.0.0.1:56290)
0805 23:11:12.550126 (+ 784us) negotiation.cc:107] Waiting for socket to connect
0805 23:11:12.550190 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0805 23:11:12.550414 (+ 224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:11:12.551178 (+ 764us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:11:12.551184 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:11:12.551221 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:11:12.551523 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:11:12.551532 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:11:12.553020 (+ 1488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:11:12.553022 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:11:12.554524 (+ 1502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:11:12.554535 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:11:12.555760 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:11:12.555764 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:11:12.555864 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:11:12.555885 (+ 21us) client_negotiation.cc:769] Sending connection context
0805 23:11:12.556419 (+ 534us) client_negotiation.cc:240] Negotiation successful
0805 23:11:12.556503 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"mutex_wait_us":224,"spinlock_wait_cycles":8064,"thread_start_us":93,"threads_started":1}
I20260805 23:11:12.557299 4501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:11:12.549731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.195:41317 (local address 127.0.0.1:55998)
0805 23:11:12.550366 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0805 23:11:12.550379 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0805 23:11:12.550539 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:11:12.551178 (+ 639us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:11:12.551184 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:11:12.551219 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:11:12.551523 (+ 304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:11:12.551532 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:11:12.553008 (+ 1476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:11:12.553012 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:11:12.554524 (+ 1512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:11:12.554535 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:11:12.556882 (+ 2347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:11:12.556886 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:11:12.556987 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:11:12.557008 (+ 21us) client_negotiation.cc:769] Sending connection context
0805 23:11:12.557088 (+ 80us) client_negotiation.cc:240] Negotiation successful
0805 23:11:12.557160 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":569,"mutex_wait_us":160,"spinlock_wait_cycles":6144,"thread_start_us":94,"threads_started":1}
I20260805 23:11:12.557574 4502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:11:12.549733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55998 (local address 127.25.92.195:41317)
0805 23:11:12.550210 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0805 23:11:12.550226 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0805 23:11:12.550458 (+ 232us) server_negotiation.cc:407] Connection header received
0805 23:11:12.550673 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:11:12.550677 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:11:12.550828 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:11:12.550954 (+ 126us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:11:12.551666 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:11:12.552868 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:11:12.555490 (+ 2622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:11:12.556748 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:11:12.556888 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:11:12.557025 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0805 23:11:12.557253 (+ 228us) server_negotiation.cc:299] Negotiation successful
0805 23:11:12.557366 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":381,"thread_start_us":126,"threads_started":1}
I20260805 23:11:12.559648 4277 tablet_service.cc:1467] Processing CreateTablet for tablet 00e7f0a0b5e24bf29781aed701119edd (DEFAULT_TABLE table=test-table [id=77a5944307b643e495eca860f49aebe0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:11:12.560170 4417 tablet_service.cc:1467] Processing CreateTablet for tablet 00e7f0a0b5e24bf29781aed701119edd (DEFAULT_TABLE table=test-table [id=77a5944307b643e495eca860f49aebe0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:11:12.560922 4277 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00e7f0a0b5e24bf29781aed701119edd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:11:12.561156 4417 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00e7f0a0b5e24bf29781aed701119edd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:11:12.561235 4129 tablet_service.cc:1467] Processing CreateTablet for tablet 00e7f0a0b5e24bf29781aed701119edd (DEFAULT_TABLE table=test-table [id=77a5944307b643e495eca860f49aebe0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:11:12.562141 4129 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00e7f0a0b5e24bf29781aed701119edd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:11:12.568104 4505 tablet_bootstrap.cc:492] T 00e7f0a0b5e24bf29781aed701119edd P d889302783554142a0a7d93e65088548: Bootstrap starting.
I20260805 23:11:12.569736 4507 tablet_bootstrap.cc:492] T 00e7f0a0b5e24bf29781aed701119edd P d8b743ce4f814feba02e7b48af42f02a: Bootstrap starting.
I20260805 23:11:12.570708 4505 tablet_bootstrap.cc:654] T 00e7f0a0b5e24bf29781aed701119edd P d889302783554142a0a7d93e65088548: Neither blocks nor log segments found. Creating new log.
I20260805 23:11:12.571617 4505 log.cc:824] T 00e7f0a0b5e24bf29781aed701119edd P d889302783554142a0a7d93e65088548: Log is configured to *not* fsync() on all Append() calls
I20260805 23:11:12.572402 4507 tablet_bootstrap.cc:654] T 00e7f0a0b5e24bf29781aed701119edd P d8b743ce4f814feba02e7b48af42f02a: Neither blocks nor log segments found. Creating new log.
I20260805 23:11:12.572491 4506 tablet_bootstrap.cc:492] T 00e7f0a0b5e24bf29781aed701119edd P 0405e56fcd704035b34b2ac37cad4597: Bootstrap starting.
I20260805 23:11:12.573247 4507 log.cc:824] T 00e7f0a0b5e24bf29781aed701119edd P d8b743ce4f814feba02e7b48af42f02a: Log is configured to *not* fsync() on all Append() calls
I20260805 23:11:12.573751 4505 tablet_bootstrap.cc:492] T 00e7f0a0b5e24bf29781aed701119edd P d889302783554142a0a7d93e65088548: No bootstrap required, opened a new log
I20260805 23:11:12.573951 4505 ts_tablet_manager.cc:1397] T 00e7f0a0b5e24bf29781aed701119edd P d889302783554142a0a7d93e65088548: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260805 23:11:12.575054 4506 tablet_bootstrap.cc:654] T 00e7f0a0b5e24bf29781aed701119edd P 0405e56fcd704035b34b2ac37cad4597: Neither blocks nor log segments found. Creating new log.
I20260805 23:11:12.575390 4507 tablet_bootstrap.cc:492] T 00e7f0a0b5e24bf29781aed701119edd P d8b743ce4f814feba02e7b48af42f02a: No bootstrap required, opened a new log
I20260805 23:11:12.575582 4507 ts_tablet_manager.cc:1397] T 00e7f0a0b5e24bf29781aed701119edd P d8b743ce4f814feba02e7b48af42f02a: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260805 23:11:12.575892 4506 log.cc:824] T 00e7f0a0b5e24bf29781aed701119edd P 0405e56fcd704035b34b2ac37cad4597: Log is configured to *not* fsync() on all Append() calls
I20260805 23:11:12.578078 4506 tablet_bootstrap.cc:492] T 00e7f0a0b5e24bf29781aed701119edd P 0405e56fcd704035b34b2ac37cad4597: No bootstrap required, opened a new log
I20260805 23:11:12.578037 4505 raft_consensus.cc:348] T 00e7f0a0b5e24bf29781aed701119edd P d889302783554142a0a7d93e65088548 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d889302783554142a0a7d93e65088548" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 39815 } } peers { permanent_uuid: "d8b743ce4f814feba02e7b48af42f02a" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 41317 } } peers { permanent_uuid: "0405e56fcd704035b34b2ac37cad4597" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 41563 } }
I20260805 23:11:12.578277 4506 ts_tablet_manager.cc:1397] T 00e7f0a0b5e24bf29781aed701119edd P 0405e56fcd704035b34b2ac37cad4597: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260805 23:11:12.578338 4505 raft_consensus.cc:374] T 00e7f0a0b5e24bf29781aed701119edd P d889302783554142a0a7d93e65088548 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:11:12.578393 4505 raft_consensus.cc:729] T 00e7f0a0b5e24bf29781aed701119edd P d889302783554142a0a7d93e65088548 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d889302783554142a0a7d93e65088548, State: Initialized, Role: FOLLOWER
I20260805 23:11:12.578826 4505 consensus_queue.cc:260] T 00e7f0a0b5e24bf29781aed701119edd P d889302783554142a0a7d93e65088548 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d889302783554142a0a7d93e65088548" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 39815 } } peers { permanent_uuid: "d8b743ce4f814feba02e7b48af42f02a" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 41317 } } peers { permanent_uuid: "0405e56fcd704035b34b2ac37cad4597" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 41563 } }
I20260805 23:11:12.579515 4204 heartbeater.cc:503] Master 127.25.92.254:35843 was elected leader, sending a full tablet report...
I20260805 23:11:12.579867 4505 ts_tablet_manager.cc:1428] T 00e7f0a0b5e24bf29781aed701119edd P d889302783554142a0a7d93e65088548: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260805 23:11:12.580603 4507 raft_consensus.cc:348] T 00e7f0a0b5e24bf29781aed701119edd P d8b743ce4f814feba02e7b48af42f02a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d889302783554142a0a7d93e65088548" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 39815 } } peers { permanent_uuid: "d8b743ce4f814feba02e7b48af42f02a" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 41317 } } peers { permanent_uuid: "0405e56fcd704035b34b2ac37cad4597" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 41563 } }
I20260805 23:11:12.580906 4507 raft_consensus.cc:374] T 00e7f0a0b5e24bf29781aed701119edd P d8b743ce4f814feba02e7b48af42f02a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:11:12.580966 4507 raft_consensus.cc:729] T 00e7f0a0b5e24bf29781aed701119edd P d8b743ce4f814feba02e7b48af42f02a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8b743ce4f814feba02e7b48af42f02a, State: Initialized, Role: FOLLOWER
I20260805 23:11:12.581446 4507 consensus_queue.cc:260] T 00e7f0a0b5e24bf29781aed701119edd P d8b743ce4f814feba02e7b48af42f02a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d889302783554142a0a7d93e65088548" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 39815 } } peers { permanent_uuid: "d8b743ce4f814feba02e7b48af42f02a" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 41317 } } peers { permanent_uuid: "0405e56fcd704035b34b2ac37cad4597" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 41563 } }
I20260805 23:11:12.582221 4488 heartbeater.cc:503] Master 127.25.92.254:35843 was elected leader, sending a full tablet report...
I20260805 23:11:12.582142 4506 raft_consensus.cc:348] T 00e7f0a0b5e24bf29781aed701119edd P 0405e56fcd704035b34b2ac37cad4597 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d889302783554142a0a7d93e65088548" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 39815 } } peers { permanent_uuid: "d8b743ce4f814feba02e7b48af42f02a" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 41317 } } peers { permanent_uuid: "0405e56fcd704035b34b2ac37cad4597" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 41563 } }
I20260805 23:11:12.582405 4506 raft_consensus.cc:374] T 00e7f0a0b5e24bf29781aed701119edd P 0405e56fcd704035b34b2ac37cad4597 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:11:12.582455 4506 raft_consensus.cc:729] T 00e7f0a0b5e24bf29781aed701119edd P 0405e56fcd704035b34b2ac37cad4597 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0405e56fcd704035b34b2ac37cad4597, State: Initialized, Role: FOLLOWER
I20260805 23:11:12.582561 4507 ts_tablet_manager.cc:1428] T 00e7f0a0b5e24bf29781aed701119edd P d8b743ce4f814feba02e7b48af42f02a: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260805 23:11:12.582880 4506 consensus_queue.cc:260] T 00e7f0a0b5e24bf29781aed701119edd P 0405e56fcd704035b34b2ac37cad4597 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d889302783554142a0a7d93e65088548" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 39815 } } peers { permanent_uuid: "d8b743ce4f814feba02e7b48af42f02a" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 41317 } } peers { permanent_uuid: "0405e56fcd704035b34b2ac37cad4597" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 41563 } }
I20260805 23:11:12.583706 4348 heartbeater.cc:503] Master 127.25.92.254:35843 was elected leader, sending a full tablet report...
I20260805 23:11:12.583905 4506 ts_tablet_manager.cc:1428] T 00e7f0a0b5e24bf29781aed701119edd P 0405e56fcd704035b34b2ac37cad4597: Time spent starting tablet: real 0.006s user 0.002s sys 0.004s
I20260805 23:11:12.624213 4511 raft_consensus.cc:482] T 00e7f0a0b5e24bf29781aed701119edd P d889302783554142a0a7d93e65088548 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 23:11:12.624495 4511 raft_consensus.cc:504] T 00e7f0a0b5e24bf29781aed701119edd P d889302783554142a0a7d93e65088548 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d889302783554142a0a7d93e65088548" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 39815 } } peers { permanent_uuid: "d8b743ce4f814feba02e7b48af42f02a" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 41317 } } peers { permanent_uuid: "0405e56fcd704035b34b2ac37cad4597" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 41563 } }
I20260805 23:11:12.624724 4512 raft_consensus.cc:482] T 00e7f0a0b5e24bf29781aed701119edd P d8b743ce4f814feba02e7b48af42f02a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 23:11:12.624955 4512 raft_consensus.cc:504] T 00e7f0a0b5e24bf29781aed701119edd P d8b743ce4f814feba02e7b48af42f02a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d889302783554142a0a7d93e65088548" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 39815 } } peers { permanent_uuid: "d8b743ce4f814feba02e7b48af42f02a" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 41317 } } peers { permanent_uuid: "0405e56fcd704035b34b2ac37cad4597" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 41563 } }
I20260805 23:11:12.626065 4512 leader_election.cc:288] T 00e7f0a0b5e24bf29781aed701119edd P d8b743ce4f814feba02e7b48af42f02a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d889302783554142a0a7d93e65088548 (127.25.92.193:39815), 0405e56fcd704035b34b2ac37cad4597 (127.25.92.194:41563)
I20260805 23:11:12.626078 4511 leader_election.cc:288] T 00e7f0a0b5e24bf29781aed701119edd P d889302783554142a0a7d93e65088548 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d8b743ce4f814feba02e7b48af42f02a (127.25.92.195:41317), 0405e56fcd704035b34b2ac37cad4597 (127.25.92.194:41563)
I20260805 23:11:12.631218 4490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:11:12.626295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.193:39815 (local address 127.25.92.195:49847)
0805 23:11:12.626497 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0805 23:11:12.626512 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0805 23:11:12.626609 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:11:12.627403 (+ 794us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:11:12.627407 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:11:12.627422 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:11:12.627509 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:11:12.627519 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:11:12.628492 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:11:12.628495 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:11:12.629832 (+ 1337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:11:12.629844 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:11:12.630744 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:11:12.630747 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:11:12.630869 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:11:12.630892 (+ 23us) client_negotiation.cc:769] Sending connection context
0805 23:11:12.630965 (+ 73us) client_negotiation.cc:240] Negotiation successful
0805 23:11:12.631054 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100}
I20260805 23:11:12.631428 4502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:11:12.626892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.193:60173 (local address 127.25.92.195:41317)
0805 23:11:12.627104 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0805 23:11:12.627109 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:11:12.627123 (+ 14us) server_negotiation.cc:407] Connection header received
0805 23:11:12.627170 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:11:12.627176 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:11:12.627225 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:11:12.627308 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:11:12.627582 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:11:12.628452 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:11:12.630114 (+ 1662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:11:12.630697 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:11:12.630805 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:11:12.630857 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0805 23:11:12.631223 (+ 366us) server_negotiation.cc:299] Negotiation successful
0805 23:11:12.631295 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":137}
I20260805 23:11:12.631573 4516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:11:12.626552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.194:41563 (local address 127.25.92.195:58787)
0805 23:11:12.627184 (+ 632us) negotiation.cc:107] Waiting for socket to connect
0805 23:11:12.627198 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0805 23:11:12.627298 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:11:12.627697 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:11:12.627699 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:11:12.627716 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:11:12.627855 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:11:12.627862 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:11:12.628929 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:11:12.628932 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:11:12.630119 (+ 1187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:11:12.630127 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:11:12.631165 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:11:12.631169 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:11:12.631265 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:11:12.631285 (+ 20us) client_negotiation.cc:769] Sending connection context
0805 23:11:12.631339 (+ 54us) client_negotiation.cc:240] Negotiation successful
0805 23:11:12.631395 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"mutex_wait_us":34,"thread_start_us":106,"threads_started":1}
I20260805 23:11:12.631778 4500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:11:12.626605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.195:49847 (local address 127.25.92.193:39815)
0805 23:11:12.626723 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0805 23:11:12.626728 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:11:12.626745 (+ 17us) server_negotiation.cc:407] Connection header received
0805 23:11:12.626795 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:11:12.626799 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:11:12.626859 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:11:12.626932 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:11:12.627647 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:11:12.628371 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:11:12.629975 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:11:12.630512 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:11:12.630630 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:11:12.630682 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0805 23:11:12.631551 (+ 869us) server_negotiation.cc:299] Negotiation successful
0805 23:11:12.631645 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260805 23:11:12.632002 4514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:11:12.626298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.195:41317 (local address 127.25.92.193:60173)
0805 23:11:12.626786 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0805 23:11:12.626799 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0805 23:11:12.626884 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:11:12.627311 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:11:12.627314 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:11:12.627325 (+ 11us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:11:12.627449 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:11:12.627455 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:11:12.628590 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:11:12.628593 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:11:12.629968 (+ 1375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:11:12.629980 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:11:12.630960 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:11:12.630963 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:11:12.631074 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:11:12.631097 (+ 23us) client_negotiation.cc:769] Sending connection context
0805 23:11:12.631634 (+ 537us) client_negotiation.cc:240] Negotiation successful
0805 23:11:12.631736 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":131,"threads_started":1}
I20260805 23:11:12.632251 4504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:11:12.626823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.193:34027 (local address 127.25.92.194:41563)
0805 23:11:12.626948 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0805 23:11:12.626952 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:11:12.627199 (+ 247us) server_negotiation.cc:407] Connection header received
0805 23:11:12.627675 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:11:12.627681 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:11:12.627728 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:11:12.627776 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:11:12.628130 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:11:12.628841 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:11:12.630293 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:11:12.630975 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:11:12.631067 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:11:12.631143 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0805 23:11:12.632007 (+ 864us) server_negotiation.cc:299] Negotiation successful
0805 23:11:12.632096 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":51,"server-negotiator.queue_time_us":51}
I20260805 23:11:12.632391 4517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:11:12.627013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.195:58787 (local address 127.25.92.194:41563)
0805 23:11:12.627361 (+ 348us) server_negotiation.cc:206] Beginning negotiation
0805 23:11:12.627372 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0805 23:11:12.627386 (+ 14us) server_negotiation.cc:407] Connection header received
0805 23:11:12.627443 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:11:12.627445 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:11:12.627480 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:11:12.627581 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:11:12.627989 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:11:12.628801 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:11:12.630240 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:11:12.631001 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:11:12.631104 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:11:12.631827 (+ 723us) server_negotiation.cc:1035] Waiting for connection context
0805 23:11:12.631995 (+ 168us) server_negotiation.cc:299] Negotiation successful
0805 23:11:12.632226 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":708,"server-negotiator.queue_time_us":273,"thread_start_us":99,"threads_started":1}
I20260805 23:11:12.632392 4153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00e7f0a0b5e24bf29781aed701119edd" candidate_uuid: "d8b743ce4f814feba02e7b48af42f02a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d889302783554142a0a7d93e65088548" is_pre_election: true
I20260805 23:11:12.632680 4153 raft_consensus.cc:2454] T 00e7f0a0b5e24bf29781aed701119edd P d889302783554142a0a7d93e65088548 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8b743ce4f814feba02e7b48af42f02a in term 0.
I20260805 23:11:12.632859 4438 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00e7f0a0b5e24bf29781aed701119edd" candidate_uuid: "d889302783554142a0a7d93e65088548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8b743ce4f814feba02e7b48af42f02a" is_pre_election: true
I20260805 23:11:12.632861 4298 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00e7f0a0b5e24bf29781aed701119edd" candidate_uuid: "d8b743ce4f814feba02e7b48af42f02a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0405e56fcd704035b34b2ac37cad4597" is_pre_election: true
I20260805 23:11:12.633060 4438 raft_consensus.cc:2454] T 00e7f0a0b5e24bf29781aed701119edd P d8b743ce4f814feba02e7b48af42f02a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d889302783554142a0a7d93e65088548 in term 0.
I20260805 23:11:12.633136 4298 raft_consensus.cc:2454] T 00e7f0a0b5e24bf29781aed701119edd P 0405e56fcd704035b34b2ac37cad4597 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8b743ce4f814feba02e7b48af42f02a in term 0.
I20260805 23:11:12.633569 4078 leader_election.cc:302] T 00e7f0a0b5e24bf29781aed701119edd P d889302783554142a0a7d93e65088548 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d889302783554142a0a7d93e65088548, d8b743ce4f814feba02e7b48af42f02a; no voters:
I20260805 23:11:12.633570 4515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:11:12.626335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.194:41563 (local address 127.25.92.193:34027)
0805 23:11:12.627112 (+ 777us) negotiation.cc:107] Waiting for socket to connect
0805 23:11:12.627126 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0805 23:11:12.627292 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:11:12.627897 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:11:12.627899 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:11:12.627916 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:11:12.628005 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:11:12.628010 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:11:12.628985 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:11:12.628988 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:11:12.630168 (+ 1180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:11:12.630177 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:11:12.631202 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:11:12.631206 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:11:12.631852 (+ 646us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:11:12.631879 (+ 27us) client_negotiation.cc:769] Sending connection context
0805 23:11:12.633315 (+ 1436us) client_negotiation.cc:240] Negotiation successful
0805 23:11:12.633403 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":709,"thread_start_us":93,"threads_started":1}
I20260805 23:11:12.633747 4369 leader_election.cc:302] T 00e7f0a0b5e24bf29781aed701119edd P d8b743ce4f814feba02e7b48af42f02a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d889302783554142a0a7d93e65088548, d8b743ce4f814feba02e7b48af42f02a; no voters:
I20260805 23:11:12.633971 4511 raft_consensus.cc:2781] T 00e7f0a0b5e24bf29781aed701119edd P d889302783554142a0a7d93e65088548 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260805 23:11:12.634083 4511 raft_consensus.cc:482] T 00e7f0a0b5e24bf29781aed701119edd P d889302783554142a0a7d93e65088548 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260805 23:11:12.634131 4511 raft_consensus.cc:3037] T 00e7f0a0b5e24bf29781aed701119edd P d889302783554142a0a7d93e65088548 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:11:12.634160 4512 raft_consensus.cc:2781] T 00e7f0a0b5e24bf29781aed701119edd P d8b743ce4f814feba02e7b48af42f02a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260805 23:11:12.634171 4298 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00e7f0a0b5e24bf29781aed701119edd" candidate_uuid: "d889302783554142a0a7d93e65088548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0405e56fcd704035b34b2ac37cad4597" is_pre_election: true
I20260805 23:11:12.634250 4512 raft_consensus.cc:482] T 00e7f0a0b5e24bf29781aed701119edd P d8b743ce4f814feba02e7b48af42f02a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260805 23:11:12.634296 4512 raft_consensus.cc:3037] T 00e7f0a0b5e24bf29781aed701119edd P d8b743ce4f814feba02e7b48af42f02a [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:11:12.634280 4298 raft_consensus.cc:2454] T 00e7f0a0b5e24bf29781aed701119edd P 0405e56fcd704035b34b2ac37cad4597 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d889302783554142a0a7d93e65088548 in term 0.
I20260805 23:11:12.635241 4511 raft_consensus.cc:504] T 00e7f0a0b5e24bf29781aed701119edd P d889302783554142a0a7d93e65088548 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d889302783554142a0a7d93e65088548" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 39815 } } peers { permanent_uuid: "d8b743ce4f814feba02e7b48af42f02a" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 41317 } } peers { permanent_uuid: "0405e56fcd704035b34b2ac37cad4597" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 41563 } }
I20260805 23:11:12.635509 4512 raft_consensus.cc:504] T 00e7f0a0b5e24bf29781aed701119edd P d8b743ce4f814feba02e7b48af42f02a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d889302783554142a0a7d93e65088548" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 39815 } } peers { permanent_uuid: "d8b743ce4f814feba02e7b48af42f02a" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 41317 } } peers { permanent_uuid: "0405e56fcd704035b34b2ac37cad4597" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 41563 } }
I20260805 23:11:12.635605 4511 leader_election.cc:288] T 00e7f0a0b5e24bf29781aed701119edd P d889302783554142a0a7d93e65088548 [CANDIDATE]: Term 1 election: Requested vote from peers d8b743ce4f814feba02e7b48af42f02a (127.25.92.195:41317), 0405e56fcd704035b34b2ac37cad4597 (127.25.92.194:41563)
I20260805 23:11:12.635865 4512 leader_election.cc:288] T 00e7f0a0b5e24bf29781aed701119edd P d8b743ce4f814feba02e7b48af42f02a [CANDIDATE]: Term 1 election: Requested vote from peers d889302783554142a0a7d93e65088548 (127.25.92.193:39815), 0405e56fcd704035b34b2ac37cad4597 (127.25.92.194:41563)
I20260805 23:11:12.635934 4438 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00e7f0a0b5e24bf29781aed701119edd" candidate_uuid: "d889302783554142a0a7d93e65088548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8b743ce4f814feba02e7b48af42f02a"
I20260805 23:11:12.636092 4438 raft_consensus.cc:2379] T 00e7f0a0b5e24bf29781aed701119edd P d8b743ce4f814feba02e7b48af42f02a [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d889302783554142a0a7d93e65088548 in current term 1: Already voted for candidate d8b743ce4f814feba02e7b48af42f02a in this term.
I20260805 23:11:12.636201 4298 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00e7f0a0b5e24bf29781aed701119edd" candidate_uuid: "d889302783554142a0a7d93e65088548" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0405e56fcd704035b34b2ac37cad4597"
I20260805 23:11:12.636317 4298 raft_consensus.cc:3037] T 00e7f0a0b5e24bf29781aed701119edd P 0405e56fcd704035b34b2ac37cad4597 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:11:12.636505 4297 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00e7f0a0b5e24bf29781aed701119edd" candidate_uuid: "d8b743ce4f814feba02e7b48af42f02a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0405e56fcd704035b34b2ac37cad4597"
I20260805 23:11:12.636637 4153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00e7f0a0b5e24bf29781aed701119edd" candidate_uuid: "d8b743ce4f814feba02e7b48af42f02a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d889302783554142a0a7d93e65088548"
I20260805 23:11:12.636798 4153 raft_consensus.cc:2379] T 00e7f0a0b5e24bf29781aed701119edd P d889302783554142a0a7d93e65088548 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d8b743ce4f814feba02e7b48af42f02a in current term 1: Already voted for candidate d889302783554142a0a7d93e65088548 in this term.
I20260805 23:11:12.637133 4369 leader_election.cc:302] T 00e7f0a0b5e24bf29781aed701119edd P d8b743ce4f814feba02e7b48af42f02a [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d8b743ce4f814feba02e7b48af42f02a; no voters: 0405e56fcd704035b34b2ac37cad4597, d889302783554142a0a7d93e65088548
I20260805 23:11:12.637298 4512 raft_consensus.cc:2726] T 00e7f0a0b5e24bf29781aed701119edd P d8b743ce4f814feba02e7b48af42f02a [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260805 23:11:12.637715 4298 raft_consensus.cc:2454] T 00e7f0a0b5e24bf29781aed701119edd P 0405e56fcd704035b34b2ac37cad4597 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d889302783554142a0a7d93e65088548 in term 1.
I20260805 23:11:12.638023 4077 leader_election.cc:302] T 00e7f0a0b5e24bf29781aed701119edd P d889302783554142a0a7d93e65088548 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0405e56fcd704035b34b2ac37cad4597, d889302783554142a0a7d93e65088548; no voters: d8b743ce4f814feba02e7b48af42f02a
I20260805 23:11:12.638204 4511 raft_consensus.cc:2781] T 00e7f0a0b5e24bf29781aed701119edd P d889302783554142a0a7d93e65088548 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:11:12.638461 4511 raft_consensus.cc:686] T 00e7f0a0b5e24bf29781aed701119edd P d889302783554142a0a7d93e65088548 [term 1 LEADER]: Becoming Leader. State: Replica: d889302783554142a0a7d93e65088548, State: Running, Role: LEADER
I20260805 23:11:12.638864 4511 consensus_queue.cc:237] T 00e7f0a0b5e24bf29781aed701119edd P d889302783554142a0a7d93e65088548 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d889302783554142a0a7d93e65088548" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 39815 } } peers { permanent_uuid: "d8b743ce4f814feba02e7b48af42f02a" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 41317 } } peers { permanent_uuid: "0405e56fcd704035b34b2ac37cad4597" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 41563 } }
I20260805 23:11:12.642078 3991 catalog_manager.cc:5495] T 00e7f0a0b5e24bf29781aed701119edd P d889302783554142a0a7d93e65088548 reported cstate change: term changed from 0 to 1, leader changed from <none> to d889302783554142a0a7d93e65088548 (127.25.92.193). New cstate: current_term: 1 leader_uuid: "d889302783554142a0a7d93e65088548" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d889302783554142a0a7d93e65088548" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 39815 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d8b743ce4f814feba02e7b48af42f02a" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 41317 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0405e56fcd704035b34b2ac37cad4597" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 41563 } health_report { overall_health: UNKNOWN } } }
I20260805 23:11:12.665730 4500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:11:12.661936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56304 (local address 127.25.92.193:39815)
0805 23:11:12.662070 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0805 23:11:12.662075 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:11:12.662098 (+ 23us) server_negotiation.cc:407] Connection header received
0805 23:11:12.662221 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:11:12.662223 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:11:12.662280 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:11:12.662379 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 23:11:12.662674 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:11:12.663242 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:11:12.664220 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:11:12.664436 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:11:12.664522 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:11:12.664796 (+ 274us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 23:11:12.665265 (+ 469us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 23:11:12.665358 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0805 23:11:12.665448 (+ 90us) server_negotiation.cc:299] Negotiation successful
0805 23:11:12.665605 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260805 23:11:12.672190 4438 raft_consensus.cc:1261] T 00e7f0a0b5e24bf29781aed701119edd P d8b743ce4f814feba02e7b48af42f02a [term 1 FOLLOWER]: Refusing update from remote peer d889302783554142a0a7d93e65088548: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:11:12.672190 4298 raft_consensus.cc:1261] T 00e7f0a0b5e24bf29781aed701119edd P 0405e56fcd704035b34b2ac37cad4597 [term 1 FOLLOWER]: Refusing update from remote peer d889302783554142a0a7d93e65088548: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:11:12.672932 4511 consensus_queue.cc:1035] T 00e7f0a0b5e24bf29781aed701119edd P d889302783554142a0a7d93e65088548 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8b743ce4f814feba02e7b48af42f02a" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 41317 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:11:12.673154 4518 consensus_queue.cc:1035] T 00e7f0a0b5e24bf29781aed701119edd P d889302783554142a0a7d93e65088548 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0405e56fcd704035b34b2ac37cad4597" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 41563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:11:12.680560 4521 mvcc.cc:205] Tried to move back new op lower bound from 7315339152057012224 to 7315339151931211776. Current Snapshot: MvccSnapshot[applied={T|T < 7315339152057012224}]
I20260805 23:11:12.684240 4525 mvcc.cc:205] Tried to move back new op lower bound from 7315339152057012224 to 7315339151931211776. Current Snapshot: MvccSnapshot[applied={T|T < 7315339152057012224}]
I20260805 23:11:12.691397 3992 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47640:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:11:12.691596 3992 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47640:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:11:12.694126 3992 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f2fd73772ea04ffa8368c49ece99103a: Sending DeleteTablet for 3 replicas of tablet 00e7f0a0b5e24bf29781aed701119edd
I20260805 23:11:12.695060 4417 tablet_service.cc:1514] Processing DeleteTablet for tablet 00e7f0a0b5e24bf29781aed701119edd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:11:12 UTC) from {username='kudu', principal='kudu/127.25.92.254@KRBTEST.COM'} at 127.0.0.1:55998
I20260805 23:11:12.695061 4277 tablet_service.cc:1514] Processing DeleteTablet for tablet 00e7f0a0b5e24bf29781aed701119edd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:11:12 UTC) from {username='kudu', principal='kudu/127.25.92.254@KRBTEST.COM'} at 127.0.0.1:43168
I20260805 23:11:12.695577 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 4059
I20260805 23:11:12.696844 4532 tablet_replica.cc:330] T 00e7f0a0b5e24bf29781aed701119edd P 0405e56fcd704035b34b2ac37cad4597: stopping tablet replica
I20260805 23:11:12.697186 4532 raft_consensus.cc:2229] T 00e7f0a0b5e24bf29781aed701119edd P 0405e56fcd704035b34b2ac37cad4597 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 23:11:12.697602 4532 raft_consensus.cc:2258] T 00e7f0a0b5e24bf29781aed701119edd P 0405e56fcd704035b34b2ac37cad4597 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:11:12.698288 4533 tablet_replica.cc:330] T 00e7f0a0b5e24bf29781aed701119edd P d8b743ce4f814feba02e7b48af42f02a: stopping tablet replica
I20260805 23:11:12.698622 4533 raft_consensus.cc:2229] T 00e7f0a0b5e24bf29781aed701119edd P d8b743ce4f814feba02e7b48af42f02a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 23:11:12.698944 4533 raft_consensus.cc:2258] T 00e7f0a0b5e24bf29781aed701119edd P d8b743ce4f814feba02e7b48af42f02a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:11:12.699373 4532 ts_tablet_manager.cc:1905] T 00e7f0a0b5e24bf29781aed701119edd P 0405e56fcd704035b34b2ac37cad4597: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:11:12.700623 4533 ts_tablet_manager.cc:1905] T 00e7f0a0b5e24bf29781aed701119edd P d8b743ce4f814feba02e7b48af42f02a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:11:12.702898 4532 ts_tablet_manager.cc:1918] T 00e7f0a0b5e24bf29781aed701119edd P 0405e56fcd704035b34b2ac37cad4597: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 23:11:12.703025 4532 log.cc:1196] T 00e7f0a0b5e24bf29781aed701119edd P 0405e56fcd704035b34b2ac37cad4597: Deleting WAL directory at /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971406251336-25971-0/minicluster-data/ts-1/wal/wals/00e7f0a0b5e24bf29781aed701119edd
I20260805 23:11:12.703395 4532 ts_tablet_manager.cc:1939] T 00e7f0a0b5e24bf29781aed701119edd P 0405e56fcd704035b34b2ac37cad4597: Deleting consensus metadata
I20260805 23:11:12.703969 4533 ts_tablet_manager.cc:1918] T 00e7f0a0b5e24bf29781aed701119edd P d8b743ce4f814feba02e7b48af42f02a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 23:11:12.704090 4533 log.cc:1196] T 00e7f0a0b5e24bf29781aed701119edd P d8b743ce4f814feba02e7b48af42f02a: Deleting WAL directory at /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971406251336-25971-0/minicluster-data/ts-2/wal/wals/00e7f0a0b5e24bf29781aed701119edd
I20260805 23:11:12.704671 4533 ts_tablet_manager.cc:1939] T 00e7f0a0b5e24bf29781aed701119edd P d8b743ce4f814feba02e7b48af42f02a: Deleting consensus metadata
W20260805 23:11:12.705041 3976 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.25.92.193:39815 (error 108)
I20260805 23:11:12.706074 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 4209
I20260805 23:11:12.706136 3976 catalog_manager.cc:4841] TS 0405e56fcd704035b34b2ac37cad4597 (127.25.92.194:41563): tablet 00e7f0a0b5e24bf29781aed701119edd (table test-table [id=77a5944307b643e495eca860f49aebe0]) successfully deleted
I20260805 23:11:12.706272 4501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:11:12.705941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.193:39815 (local address 127.0.0.1:56306)
0805 23:11:12.706066 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0805 23:11:12.706138 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.193:39815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260805 23:11:12.706386 3976 catalog_manager.cc:4584] TS d889302783554142a0a7d93e65088548 (127.25.92.193:39815): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 00e7f0a0b5e24bf29781aed701119edd: Network error: Client connection negotiation failed: client connection to 127.25.92.193:39815: connect: Connection refused (error 111)
I20260805 23:11:12.710141 3977 catalog_manager.cc:4841] TS d8b743ce4f814feba02e7b48af42f02a (127.25.92.195:41317): tablet 00e7f0a0b5e24bf29781aed701119edd (table test-table [id=77a5944307b643e495eca860f49aebe0]) successfully deleted
I20260805 23:11:12.713016 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 4353
I20260805 23:11:12.719844 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 3959
2026-08-05T23:11:12Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2439 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 23:11:12 dist-test-slave-ss4n krb5kdc[4537](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 23:11:12 dist-test-slave-ss4n krb5kdc[4537](info): set up 2 sockets
Aug 05 23:11:12 dist-test-slave-ss4n krb5kdc[4537](info): commencing operation
krb5kdc: starting...
W20260805 23:11:13.839154 25971 mini_kdc.cc:121] Time spent starting KDC: real 1.096s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/krb5kdc/test-user.keytab.
Aug 05 23:11:13 dist-test-slave-ss4n krb5kdc[4537](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971473, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T23:11:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:11:13Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.25.92.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.25.92.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.25.92.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/krb5kdc/rangeradmin_127.25.92.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.25.92.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/krb5kdc/rangeradmin_127.25.92.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.25.92.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.25.92.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.25.92.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/krb5kdc/rangerlookup_127.25.92.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.25.92.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/krb5kdc/rangerlookup_127.25.92.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.25.92.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.25.92.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.25.92.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/krb5kdc/HTTP_127.25.92.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.25.92.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/krb5kdc/HTTP_127.25.92.212@KRBTEST.COM.keytab.
I20260805 23:11:14.037798 25971 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskd06taO/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/minicluster-data/postgres -l logfile start
2026-08-05 23:11:17.182 UTC [4569] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-05 23:11:17.182 UTC [4569] LOG: listening on IPv4 address "127.25.92.212", port 45785
2026-08-05 23:11:17.187 UTC [4569] LOG: listening on Unix socket "/tmp/.s.PGSQL.45785"
2026-08-05 23:11:17.211 UTC [4572] LOG: database system was shut down at 2026-08-05 23:11:15 UTC
2026-08-05 23:11:17.216 UTC [4569] LOG: database system is ready to accept connections
I20260805 23:11:18.184662 25971 mini_postgres.cc:96] Postgres bound to 45785
2026-08-05 23:11:18.192 UTC [4580] FATAL: database "slave" does not exist
127.25.92.212:45785 - accepting connections
I20260805 23:11:18.193360 25971 mini_ranger.cc:161] Starting Ranger...
I20260805 23:11:18.212715 25971 mini_ranger.cc:84] Created miniranger Postgres user
I20260805 23:11:18.847769 25971 mini_ranger.cc:87] Created ranger Postgres database
I20260805 23:11:18.847882 25971 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/minicluster-data/ranger-admin
2026-08-05 23:11:19,199 [I] DB FLAVOR :POSTGRES
2026-08-05 23:11:19,200 [I] --------- Verifying Ranger DB connection ---------
2026-08-05 23:11:19,200 [I] Checking connection..
2026-08-05 23:11:19,200 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskd06taO/build/debug/bin/postgresql.jar:/tmp/dist-test-taskd06taO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.92.212:45785/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-05 23:11:19,459 [I] Checking connection passed.
2026-08-05 23:11:19,460 [I] --------- Verifying version history table ---------
2026-08-05 23:11:19,460 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskd06taO/build/debug/bin/postgresql.jar:/tmp/dist-test-taskd06taO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.92.212:45785/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-05 23:11:19,723 [I] Table x_db_version_h does not exist in database ranger
2026-08-05 23:11:19,723 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskd06taO/build/debug/bin/postgresql.jar:/tmp/dist-test-taskd06taO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.92.212:45785/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-05 23:11:20,011 [I] Table x_db_version_h does not exist in database ranger
2026-08-05 23:11:20,012 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-05 23:11:20,012 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskd06taO/build/debug/bin/postgresql.jar:/tmp/dist-test-taskd06taO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.92.212:45785/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskd06taO/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-05 23:11:20.296 UTC [4648] WARNING: there is no transaction in progress
2026-08-05 23:11:20,303 [I] create_dbversion_catalog.sql file imported successfully
2026-08-05 23:11:20,304 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskd06taO/build/debug/bin/postgresql.jar:/tmp/dist-test-taskd06taO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.92.212:45785/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-05 23:11:20,576 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-05 23:11:20,577 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-05 23:11:20,577 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskd06taO/build/debug/bin/postgresql.jar:/tmp/dist-test-taskd06taO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.92.212:45785/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-05 23:11:20,850 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskd06taO/build/debug/bin/postgresql.jar:/tmp/dist-test-taskd06taO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.92.212:45785/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-05 23:11:21,107 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskd06taO/build/debug/bin/postgresql.jar:/tmp/dist-test-taskd06taO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.92.212:45785/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-ss4n.c.gcp-upstream.internal','N') ;"
2026-08-05 23:11:21,338 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskd06taO/build/debug/bin/postgresql.jar:/tmp/dist-test-taskd06taO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.92.212:45785/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-05 23:11:21,622 [I] Table x_portal_user does not exist in database ranger
2026-08-05 23:11:21,622 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskd06taO/build/debug/bin/postgresql.jar:/tmp/dist-test-taskd06taO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.92.212:45785/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-05 23:11:21,876 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-05 23:11:21,876 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskd06taO/build/debug/bin/postgresql.jar:/tmp/dist-test-taskd06taO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.92.212:45785/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-05 23:11:22,122 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskd06taO/build/debug/bin/postgresql.jar:/tmp/dist-test-taskd06taO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.92.212:45785/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-05 23:11:22,389 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-05 23:11:22,389 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskd06taO/build/debug/bin/postgresql.jar:/tmp/dist-test-taskd06taO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.92.212:45785/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskd06taO/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-05 23:11:23.539 UTC [4761] WARNING: there is no transaction in progress
2026-08-05 23:11:23.561 UTC [4761] WARNING: there is no transaction in progress
2026-08-05 23:11:23.583 UTC [4761] WARNING: there is no transaction in progress
2026-08-05 23:11:23.602 UTC [4761] WARNING: there is no transaction in progress
2026-08-05 23:11:23.625 UTC [4761] WARNING: there is no transaction in progress
2026-08-05 23:11:23.652 UTC [4761] WARNING: there is no transaction in progress
2026-08-05 23:11:23.733 UTC [4761] WARNING: there is no transaction in progress
2026-08-05 23:11:23.742 UTC [4761] WARNING: there is no transaction in progress
2026-08-05 23:11:23.760 UTC [4761] WARNING: there is no transaction in progress
2026-08-05 23:11:23.773 UTC [4761] WARNING: there is no transaction in progress
2026-08-05 23:11:23.785 UTC [4761] WARNING: there is no transaction in progress
2026-08-05 23:11:23.802 UTC [4761] WARNING: there is no transaction in progress
2026-08-05 23:11:23.814 UTC [4761] WARNING: there is no transaction in progress
2026-08-05 23:11:23.823 UTC [4761] WARNING: there is no transaction in progress
2026-08-05 23:11:23.832 UTC [4761] WARNING: there is no transaction in progress
2026-08-05 23:11:23.842 UTC [4761] WARNING: there is no transaction in progress
2026-08-05 23:11:24,693 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-05 23:11:24,694 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskd06taO/build/debug/bin/postgresql.jar:/tmp/dist-test-taskd06taO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.92.212:45785/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-05 23:11:24,951 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-05 23:11:24,951 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskd06taO/build/debug/bin/postgresql.jar:/tmp/dist-test-taskd06taO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.92.212:45785/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-05 23:11:25,244 [I] Table x_portal_user already exists in database 'ranger'
2026-08-05 23:11:25,244 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskd06taO/build/debug/bin/postgresql.jar:/tmp/dist-test-taskd06taO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.92.212:45785/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-05 23:11:25,522 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-05 23:11:25,523 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskd06taO/build/debug/bin/postgresql.jar:/tmp/dist-test-taskd06taO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.92.212:45785/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-05 23:11:25,805 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskd06taO/build/debug/bin/postgresql.jar:/tmp/dist-test-taskd06taO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.92.212:45785/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-05 23:11:26,083 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskd06taO/build/debug/bin/postgresql.jar:/tmp/dist-test-taskd06taO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.92.212:45785/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-ss4n.c.gcp-upstream.internal';"
2026-08-05 23:11:26,353 [I] CORE_DB_SCHEMA import status has been updated
2026-08-05 23:11:26,353 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskd06taO/build/debug/bin/postgresql.jar:/tmp/dist-test-taskd06taO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.92.212:45785/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-05 23:11:26,627 [I] DB_PATCHES have already been applied
I20260805 23:11:26.633924 25971 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/minicluster-data/ranger-admin:/tmp/dist-test-taskd06taO/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskd06taO/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskd06taO/build/debug/bin/postgresql.jar:/tmp/dist-test-taskd06taO/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260805 23:11:26.634045 25971 mini_ranger.cc:193] Using host: 127.25.92.212
I20260805 23:11:26.637952 25971 mini_ranger.cc:227] Ranger admin URL: http://127.25.92.212:45953
Aug 05, 2026 11:11:26 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 05, 2026 11:11:26 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 05, 2026 11:11:27 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/minicluster-data/ranger-admin/ews/webapp
Aug 05, 2026 11:11:27 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 05, 2026 11:11:27 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 05, 2026 11:11:27 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/minicluster-data/ranger-admin/ews/webapp].
Aug 05, 2026 11:11:27 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260805 23:11:27.800793 25971 mini_ranger.cc:160] Time spent starting Ranger: real 9.607s user 0.002s sys 0.012s
Aug 05, 2026 11:11:49 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 05, 2026 11:11:50 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 05, 2026 11:11:50 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 05, 2026 11:11:50 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 05, 2026 11:11:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 05, 2026 11:11:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 05, 2026 11:11:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 05, 2026 11:11:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 05, 2026 11:11:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 05, 2026 11:11:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 05, 2026 11:11:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 05, 2026 11:11:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 05, 2026 11:11:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 05, 2026 11:11:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 05, 2026 11:11:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 05, 2026 11:11:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 05, 2026 11:11:50 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 05, 2026 11:11:50 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260805 23:11:51.580008 25971 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.25.92.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.25.92.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.25.92.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/krb5kdc/rangerkms_127.25.92.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.25.92.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/krb5kdc/rangerkms_127.25.92.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.25.92.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.25.92.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.25.92.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/krb5kdc/HTTP_127.25.92.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.25.92.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/krb5kdc/HTTP_127.25.92.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 05 23:11:51 dist-test-slave-ss4n krb5kdc[4537](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971511, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260805 23:11:51.693905 25971 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260805 23:11:51.706547 25971 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260805 23:11:52.753641 25971 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260805 23:11:52.753756 25971 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/minicluster-data/ranger-kms
I20260805 23:11:52.753885 25971 mini_ranger_kms.cc:227] Using postgres at 127.25.92.212:45785
2026-08-05 23:11:53,336 [I] DB FLAVOR :POSTGRES
2026-08-05 23:11:53,337 [I] --------- Verifying Ranger DB connection ---------
2026-08-05 23:11:53,337 [I] Checking connection
2026-08-05 23:11:53,337 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskd06taO/build/debug/bin/postgresql.jar:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.92.212:45785/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-05 23:11:53,591 [I] connection success
2026-08-05 23:11:53,591 [I] --------- Verifying Ranger DB tables ---------
2026-08-05 23:11:53,591 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-05 23:11:53,591 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskd06taO/build/debug/bin/postgresql.jar:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.92.212:45785/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-05 23:11:53,831 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-05 23:11:53,831 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-05 23:11:53,832 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-05 23:11:53,832 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskd06taO/build/debug/bin/postgresql.jar:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.92.212:45785/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-05 23:11:54,115 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260805 23:11:54.658869 25971 mini_ranger_kms.cc:310] Created kms service
I20260805 23:11:54.901065 25971 mini_ranger_kms.cc:326] Created kudu user
I20260805 23:11:54.946499 25971 mini_ranger_kms.cc:343] Created rangerkms user
Aug 05 23:11:54 dist-test-slave-ss4n krb5kdc[4537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971511, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.25.92.212@KRBTEST.COM
I20260805 23:11:55.341457 25971 mini_ranger_kms.cc:384] Added ranger policy
I20260805 23:11:55.341670 25971 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/minicluster-data/ranger-kms:/tmp/dist-test-taskd06taO/build/debug/bin/postgresql.jar:/tmp/dist-test-taskd06taO/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskd06taO/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260805 23:11:55.341697 25971 mini_ranger_kms.cc:240] Using host: 127.25.92.212
I20260805 23:11:55.346956 25971 mini_ranger_kms.cc:276] Ranger KMS PID: 4955
I20260805 23:11:55.347107 25971 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.25.92.212:43995
Aug 05, 2026 11:11:56 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 05, 2026 11:11:56 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 05, 2026 11:11:56 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 05, 2026 11:11:56 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 05, 2026 11:11:56 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/minicluster-data/ranger-kms/ews/webapp].
Aug 05, 2026 11:11:56 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 05 23:11:57 dist-test-slave-ss4n krb5kdc[4537](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1785971517, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.25.92.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:11:57.809226 25971 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.115s user 0.019s sys 0.300s
I20260805 23:11:57.809371 25971 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260805 23:11:57.809520 25971 mini_ranger_kms.cc:401] 127.25.92.212:43995/kms/v1/keys
Aug 05 23:11:58 dist-test-slave-ss4n krb5kdc[4537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971517, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.25.92.212@KRBTEST.COM for HTTP/127.25.92.212@KRBTEST.COM
Aug 05 23:11:58 dist-test-slave-ss4n krb5kdc[4537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971517, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.25.92.212@KRBTEST.COM for HTTP/127.25.92.212@KRBTEST.COM
Aug 05 23:12:00 dist-test-slave-ss4n krb5kdc[4537](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971520, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.25.92.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.92.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.92.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab.
Entry for principal kudu/127.25.92.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab.
I20260805 23:12:00.890935 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.254:37969
--webserver_interface=127.25.92.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab
--principal=kudu/127.25.92.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:38245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.92.254:37969
--ranger_config_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.25.92.212:43995/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:12:01.012056 5011 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:12:01.012380 5011 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:12:01.012483 5011 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:12:01.016727 5011 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:12:01.016839 5011 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:12:01.016880 5011 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:12:01.016939 5011 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:12:01.016973 5011 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:12:01.022797 5011 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:38245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/minicluster-data/master-0/wal
--ranger_kms_url=127.25.92.212:43995/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.25.92.254:37969
--ranger_config_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab
--principal=kudu/127.25.92.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.254:37969
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.92.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5011
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:12:01.023937 5011 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:12:01.024982 5011 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:12:01.034991 5017 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:12:01.034992 5018 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:12:01.035161 5020 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:12:01.035329 5011 server_base.cc:1034] running on GCE node
I20260805 23:12:01.036332 5011 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:12:01.037818 5011 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:12:01.039036 5011 hybrid_clock.cc:648] HybridClock initialized: now 1785971521039020 us; error 38 us; skew 500 ppm
Aug 05 23:12:01 dist-test-slave-ss4n krb5kdc[4537](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971521, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:12:01.045367 5011 init.cc:377] Logged in from keytab as kudu/127.25.92.254@KRBTEST.COM (short username kudu)
I20260805 23:12:01.046530 5011 webserver.cc:460] Webserver started at http://127.25.92.254:36979/ using document root <none> and password file <none>
I20260805 23:12:01.047127 5011 fs_manager.cc:359] Metadata directory not provided
I20260805 23:12:01.047186 5011 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:12:01.047413 5011 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 05 23:12:01 dist-test-slave-ss4n krb5kdc[4537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971521, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.254@KRBTEST.COM for HTTP/127.25.92.212@KRBTEST.COM
I20260805 23:12:01.111591 5011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/minicluster-data/master-0/data/instance:
uuid: "66baab3ba0284421a220eed88f7089da"
format_stamp: "Formatted at 2026-08-05 23:12:01 on dist-test-slave-ss4n"
server_key: "0bbc0ce82a168596db0ed70b5531031e"
server_key_iv: "bf532763608a640da20074fe2dcf0f8a"
server_key_version: "kuduclusterkey@0"
I20260805 23:12:01.112200 5011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/minicluster-data/master-0/wal/instance:
uuid: "66baab3ba0284421a220eed88f7089da"
format_stamp: "Formatted at 2026-08-05 23:12:01 on dist-test-slave-ss4n"
server_key: "0bbc0ce82a168596db0ed70b5531031e"
server_key_iv: "bf532763608a640da20074fe2dcf0f8a"
server_key_version: "kuduclusterkey@0"
I20260805 23:12:01.116113 5011 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260805 23:12:01.133204 5028 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:12:01.134629 5011 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260805 23:12:01.134773 5011 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/minicluster-data/master-0/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/minicluster-data/master-0/wal
uuid: "66baab3ba0284421a220eed88f7089da"
format_stamp: "Formatted at 2026-08-05 23:12:01 on dist-test-slave-ss4n"
server_key: "0bbc0ce82a168596db0ed70b5531031e"
server_key_iv: "bf532763608a640da20074fe2dcf0f8a"
server_key_version: "kuduclusterkey@0"
I20260805 23:12:01.134904 5011 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:12:01.152382 5011 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:12:01.155653 5011 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:12:01.155871 5011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:12:01.164217 5011 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.254:37969
I20260805 23:12:01.164268 5082 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.254:37969 every 8 connection(s)
I20260805 23:12:01.165314 5011 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
I20260805 23:12:01.168053 5083 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:12:01.170091 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 5011
I20260805 23:12:01.170217 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/minicluster-data/master-0/wal/instance
I20260805 23:12:01.170511 25971 external_mini_cluster.cc:1428] Setting key 219626c2003cafbcf124fd217f1b2934
I20260805 23:12:01.175194 5083 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66baab3ba0284421a220eed88f7089da: Bootstrap starting.
I20260805 23:12:01.177737 5083 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 66baab3ba0284421a220eed88f7089da: Neither blocks nor log segments found. Creating new log.
I20260805 23:12:01.178628 5083 log.cc:824] T 00000000000000000000000000000000 P 66baab3ba0284421a220eed88f7089da: Log is configured to *not* fsync() on all Append() calls
Aug 05 23:12:01 dist-test-slave-ss4n krb5kdc[4537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971520, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM
I20260805 23:12:01.180994 5083 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66baab3ba0284421a220eed88f7089da: No bootstrap required, opened a new log
I20260805 23:12:01.184254 5083 raft_consensus.cc:348] T 00000000000000000000000000000000 P 66baab3ba0284421a220eed88f7089da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66baab3ba0284421a220eed88f7089da" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 37969 } }
I20260805 23:12:01.184566 5083 raft_consensus.cc:374] T 00000000000000000000000000000000 P 66baab3ba0284421a220eed88f7089da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:12:01.184661 5083 raft_consensus.cc:729] T 00000000000000000000000000000000 P 66baab3ba0284421a220eed88f7089da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66baab3ba0284421a220eed88f7089da, State: Initialized, Role: FOLLOWER
I20260805 23:12:01.185133 5083 consensus_queue.cc:260] T 00000000000000000000000000000000 P 66baab3ba0284421a220eed88f7089da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66baab3ba0284421a220eed88f7089da" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 37969 } }
I20260805 23:12:01.185289 5083 raft_consensus.cc:388] T 00000000000000000000000000000000 P 66baab3ba0284421a220eed88f7089da [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:12:01.185372 5083 raft_consensus.cc:482] T 00000000000000000000000000000000 P 66baab3ba0284421a220eed88f7089da [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:12:01.185513 5083 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 66baab3ba0284421a220eed88f7089da [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:12:01.186714 5083 raft_consensus.cc:504] T 00000000000000000000000000000000 P 66baab3ba0284421a220eed88f7089da [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66baab3ba0284421a220eed88f7089da" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 37969 } }
I20260805 23:12:01.187109 5083 leader_election.cc:302] T 00000000000000000000000000000000 P 66baab3ba0284421a220eed88f7089da [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 66baab3ba0284421a220eed88f7089da; no voters:
I20260805 23:12:01.187458 5083 leader_election.cc:288] T 00000000000000000000000000000000 P 66baab3ba0284421a220eed88f7089da [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 23:12:01.187597 5088 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 66baab3ba0284421a220eed88f7089da [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:12:01.187834 5088 raft_consensus.cc:686] T 00000000000000000000000000000000 P 66baab3ba0284421a220eed88f7089da [term 1 LEADER]: Becoming Leader. State: Replica: 66baab3ba0284421a220eed88f7089da, State: Running, Role: LEADER
I20260805 23:12:01.188182 5088 consensus_queue.cc:237] T 00000000000000000000000000000000 P 66baab3ba0284421a220eed88f7089da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66baab3ba0284421a220eed88f7089da" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 37969 } }
I20260805 23:12:01.188793 5083 sys_catalog.cc:564] T 00000000000000000000000000000000 P 66baab3ba0284421a220eed88f7089da [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:12:01.189157 5086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:12:01.171932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44224 (local address 127.25.92.254:37969)
0805 23:12:01.172370 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0805 23:12:01.172390 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0805 23:12:01.172434 (+ 44us) server_negotiation.cc:407] Connection header received
0805 23:12:01.173316 (+ 882us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:12:01.173341 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:12:01.173802 (+ 461us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:12:01.174172 (+ 370us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:12:01.174983 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:12:01.176426 (+ 1443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:12:01.177635 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:12:01.178320 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:12:01.178504 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:12:01.181438 (+ 2934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:12:01.181444 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:12:01.181460 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:12:01.181519 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:12:01.185280 (+ 3761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:12:01.185821 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:12:01.185829 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:12:01.185834 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:12:01.185963 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:12:01.186253 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:12:01.186257 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:12:01.186259 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:12:01.186823 (+ 564us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:12:01.187012 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0805 23:12:01.187974 (+ 962us) server_negotiation.cc:299] Negotiation successful
0805 23:12:01.188247 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":123,"threads_started":1}
I20260805 23:12:01.191677 5088 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66baab3ba0284421a220eed88f7089da [sys.catalog]: SysCatalogTable state changed. Reason: New leader 66baab3ba0284421a220eed88f7089da. Latest consensus state: current_term: 1 leader_uuid: "66baab3ba0284421a220eed88f7089da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66baab3ba0284421a220eed88f7089da" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 37969 } } }
I20260805 23:12:01.191869 5088 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66baab3ba0284421a220eed88f7089da [sys.catalog]: This master's current role is: LEADER
I20260805 23:12:01.192271 5103 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:12:01.192654 5083 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260805 23:12:01.194320 5089 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66baab3ba0284421a220eed88f7089da [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "66baab3ba0284421a220eed88f7089da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66baab3ba0284421a220eed88f7089da" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 37969 } } }
I20260805 23:12:01.194422 5089 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66baab3ba0284421a220eed88f7089da [sys.catalog]: This master's current role is: LEADER
I20260805 23:12:01.195959 5103 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:12:01.207824 5103 catalog_manager.cc:1347] Generated new cluster ID: 2d86af7a95b54387b52937eb847e00b0
I20260805 23:12:01.208019 5103 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:12:01.235612 5103 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:12:01.237020 5103 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:12:01.244903 5103 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 66baab3ba0284421a220eed88f7089da: Generated new TSK 0
I20260805 23:12:01.245868 5103 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.92.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.92.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.92.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.193.keytab.
Entry for principal kudu/127.25.92.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.193.keytab.
I20260805 23:12:02.049801 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.25.92.212:43995/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.92.193:0
--local_ip_for_outbound_sockets=127.25.92.193
--webserver_interface=127.25.92.193
--webserver_port=0
--tserver_master_addrs=127.25.92.254:37969
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.193.keytab
--principal=kudu/127.25.92.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:38245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:12:02.183188 5130 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:12:02.183526 5130 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:12:02.183655 5130 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:12:02.188640 5130 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:12:02.188728 5130 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:12:02.188841 5130 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.92.193
I20260805 23:12:02.195526 5130 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:38245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.25.92.212:43995/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.193.keytab
--principal=kudu/127.25.92.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.92.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.92.193
--webserver_port=0
--tserver_master_addrs=127.25.92.254:37969
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5130
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.92.193
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:12:02.196797 5130 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:12:02.198153 5130 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:12:02.212030 5135 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:12:02.212038 5139 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:12:02.213106 5130 server_base.cc:1034] running on GCE node
W20260805 23:12:02.214574 5137 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:12:02.215098 5130 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:12:02.215867 5130 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:12:02.217065 5130 hybrid_clock.cc:648] HybridClock initialized: now 1785971522217026 us; error 53 us; skew 500 ppm
Aug 05 23:12:02 dist-test-slave-ss4n krb5kdc[4537](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971522, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:12:02.220280 5130 init.cc:377] Logged in from keytab as kudu/127.25.92.193@KRBTEST.COM (short username kudu)
I20260805 23:12:02.221339 5130 webserver.cc:460] Webserver started at http://127.25.92.193:45339/ using document root <none> and password file <none>
I20260805 23:12:02.222098 5130 fs_manager.cc:359] Metadata directory not provided
I20260805 23:12:02.222218 5130 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:12:02.222489 5130 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 05 23:12:02 dist-test-slave-ss4n krb5kdc[4537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971522, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.193@KRBTEST.COM for HTTP/127.25.92.212@KRBTEST.COM
I20260805 23:12:02.238034 5130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/minicluster-data/ts-0/data/instance:
uuid: "a9cd57e8a78a4d4cb221c5c154b93a8e"
format_stamp: "Formatted at 2026-08-05 23:12:02 on dist-test-slave-ss4n"
server_key: "e88cd02d1dd1c7a57c2a03480027c67d"
server_key_iv: "b67f7c62b3d6a93099a0911e49d75c13"
server_key_version: "kuduclusterkey@0"
I20260805 23:12:02.239130 5130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/minicluster-data/ts-0/wal/instance:
uuid: "a9cd57e8a78a4d4cb221c5c154b93a8e"
format_stamp: "Formatted at 2026-08-05 23:12:02 on dist-test-slave-ss4n"
server_key: "e88cd02d1dd1c7a57c2a03480027c67d"
server_key_iv: "b67f7c62b3d6a93099a0911e49d75c13"
server_key_version: "kuduclusterkey@0"
I20260805 23:12:02.245210 5130 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260805 23:12:02.266897 5150 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:12:02.268746 5130 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.001s sys 0.000s
I20260805 23:12:02.268893 5130 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/minicluster-data/ts-0/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/minicluster-data/ts-0/wal
uuid: "a9cd57e8a78a4d4cb221c5c154b93a8e"
format_stamp: "Formatted at 2026-08-05 23:12:02 on dist-test-slave-ss4n"
server_key: "e88cd02d1dd1c7a57c2a03480027c67d"
server_key_iv: "b67f7c62b3d6a93099a0911e49d75c13"
server_key_version: "kuduclusterkey@0"
I20260805 23:12:02.269043 5130 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:12:02.292517 5130 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:12:02.296512 5130 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:12:02.296845 5130 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:12:02.297698 5130 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:12:02.298938 5130 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:12:02.299049 5130 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:12:02.299185 5130 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:12:02.299271 5130 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:12:02.311335 5130 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.193:42925
I20260805 23:12:02.312716 5130 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/minicluster-data/ts-0/data/info.pb
I20260805 23:12:02.315421 5273 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.193:42925 every 8 connection(s)
I20260805 23:12:02.318642 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 5130
I20260805 23:12:02.318763 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/minicluster-data/ts-0/wal/instance
I20260805 23:12:02.319103 25971 external_mini_cluster.cc:1428] Setting key c2a6fa0737fbed8f560029622a0dec57
Aug 05 23:12:02 dist-test-slave-ss4n krb5kdc[4537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971522, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.193@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM
I20260805 23:12:02.334779 5279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:12:02.317985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:37969 (local address 127.25.92.193:46003)
0805 23:12:02.319415 (+ 1430us) negotiation.cc:107] Waiting for socket to connect
0805 23:12:02.319480 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0805 23:12:02.320600 (+ 1120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:12:02.321397 (+ 797us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:12:02.321409 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:12:02.321823 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:12:02.322586 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:12:02.322610 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:12:02.323476 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:12:02.323482 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:12:02.324897 (+ 1415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:12:02.324908 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:12:02.325754 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:12:02.325758 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:12:02.325967 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:12:02.326703 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:12:02.326726 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:12:02.328545 (+ 1819us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:12:02.331016 (+ 2471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:12:02.331037 (+ 21us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:12:02.331058 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:12:02.331480 (+ 422us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:12:02.331844 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:12:02.331849 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:12:02.331853 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:12:02.332112 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:12:02.332674 (+ 562us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:12:02.332684 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:12:02.333051 (+ 367us) client_negotiation.cc:769] Sending connection context
0805 23:12:02.333278 (+ 227us) client_negotiation.cc:240] Negotiation successful
0805 23:12:02.333665 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1197,"thread_start_us":161,"threads_started":1}
I20260805 23:12:02.335660 5278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:12:02.317919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.193:46003 (local address 127.25.92.254:37969)
0805 23:12:02.318331 (+ 412us) server_negotiation.cc:206] Beginning negotiation
0805 23:12:02.318336 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:12:02.319727 (+ 1391us) server_negotiation.cc:407] Connection header received
0805 23:12:02.320932 (+ 1205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:12:02.320935 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:12:02.320987 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:12:02.321079 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:12:02.322782 (+ 1703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:12:02.323341 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:12:02.325066 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:12:02.325414 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:12:02.325556 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:12:02.328774 (+ 3218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:12:02.328778 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:12:02.328781 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:12:02.328810 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:12:02.330854 (+ 2044us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:12:02.331621 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:12:02.331626 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:12:02.331630 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:12:02.331734 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:12:02.332238 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:12:02.332242 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:12:02.332246 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:12:02.332512 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:12:02.332613 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0805 23:12:02.335315 (+ 2702us) server_negotiation.cc:299] Negotiation successful
0805 23:12:02.335485 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":96,"threads_started":1}
I20260805 23:12:02.337703 5274 heartbeater.cc:347] Connected to a master server at 127.25.92.254:37969
I20260805 23:12:02.337978 5274 heartbeater.cc:464] Registering TS with master...
I20260805 23:12:02.338675 5274 heartbeater.cc:511] Master 127.25.92.254:37969 requested a full tablet report, sending...
I20260805 23:12:02.340720 5042 ts_manager.cc:194] Registered new tserver with Master: a9cd57e8a78a4d4cb221c5c154b93a8e (127.25.92.193:42925)
I20260805 23:12:02.342362 5042 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.92.193@KRBTEST.COM'} at 127.25.92.193:46003
WARNING: no policy specified for kudu/127.25.92.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.92.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.92.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.194.keytab.
Entry for principal kudu/127.25.92.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.194.keytab.
I20260805 23:12:02.374818 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.25.92.212:43995/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.92.194:0
--local_ip_for_outbound_sockets=127.25.92.194
--webserver_interface=127.25.92.194
--webserver_port=0
--tserver_master_addrs=127.25.92.254:37969
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.194.keytab
--principal=kudu/127.25.92.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:38245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 05 23:12:02 dist-test-slave-ss4n krb5kdc[4537](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1785971522, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260805 23:12:02.540598 5282 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:12:02.541109 5282 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:12:02.541240 5282 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:12:02.547832 5282 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:12:02.547999 5282 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:12:02.548128 5282 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.92.194
I20260805 23:12:02.556880 5282 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:38245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.25.92.212:43995/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.194.keytab
--principal=kudu/127.25.92.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.92.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.92.194
--webserver_port=0
--tserver_master_addrs=127.25.92.254:37969
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5282
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.92.194
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:12:02.558631 5282 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:12:02.560189 5282 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:12:02.576123 5289 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:12:02.576331 5282 server_base.cc:1034] running on GCE node
W20260805 23:12:02.576653 5291 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:12:02.576774 5288 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:12:02.577410 5282 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:12:02.578402 5282 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:12:02.579630 5282 hybrid_clock.cc:648] HybridClock initialized: now 1785971522579602 us; error 46 us; skew 500 ppm
Aug 05 23:12:02 dist-test-slave-ss4n krb5kdc[4537](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971522, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:12:02.584028 5282 init.cc:377] Logged in from keytab as kudu/127.25.92.194@KRBTEST.COM (short username kudu)
I20260805 23:12:02.585417 5282 webserver.cc:460] Webserver started at http://127.25.92.194:35403/ using document root <none> and password file <none>
I20260805 23:12:02.586308 5282 fs_manager.cc:359] Metadata directory not provided
I20260805 23:12:02.586422 5282 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:12:02.586704 5282 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 05 23:12:02 dist-test-slave-ss4n krb5kdc[4537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971522, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.194@KRBTEST.COM for HTTP/127.25.92.212@KRBTEST.COM
I20260805 23:12:02.616274 5282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/minicluster-data/ts-1/data/instance:
uuid: "3c6061a044a24b609169f6c25b8a9819"
format_stamp: "Formatted at 2026-08-05 23:12:02 on dist-test-slave-ss4n"
server_key: "a3f52307121ca0420c7cf13d2f66c91b"
server_key_iv: "bd6e69f0cb87f0b9df04432ed0f4141b"
server_key_version: "kuduclusterkey@0"
I20260805 23:12:02.617200 5282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/minicluster-data/ts-1/wal/instance:
uuid: "3c6061a044a24b609169f6c25b8a9819"
format_stamp: "Formatted at 2026-08-05 23:12:02 on dist-test-slave-ss4n"
server_key: "a3f52307121ca0420c7cf13d2f66c91b"
server_key_iv: "bd6e69f0cb87f0b9df04432ed0f4141b"
server_key_version: "kuduclusterkey@0"
I20260805 23:12:02.623543 5282 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.001s
I20260805 23:12:02.650658 5299 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:12:02.655313 5282 fs_manager.cc:725] Time spent opening block manager: real 0.009s user 0.002s sys 0.000s
I20260805 23:12:02.655512 5282 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/minicluster-data/ts-1/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/minicluster-data/ts-1/wal
uuid: "3c6061a044a24b609169f6c25b8a9819"
format_stamp: "Formatted at 2026-08-05 23:12:02 on dist-test-slave-ss4n"
server_key: "a3f52307121ca0420c7cf13d2f66c91b"
server_key_iv: "bd6e69f0cb87f0b9df04432ed0f4141b"
server_key_version: "kuduclusterkey@0"
I20260805 23:12:02.655679 5282 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:12:02.673192 5282 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:12:02.683363 5282 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:12:02.683658 5282 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:12:02.684409 5282 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:12:02.685568 5282 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:12:02.685674 5282 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:12:02.685801 5282 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:12:02.685863 5282 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:12:02.697134 5282 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.194:43897
I20260805 23:12:02.698392 5282 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/minicluster-data/ts-1/data/info.pb
I20260805 23:12:02.699397 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 5282
I20260805 23:12:02.699517 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/minicluster-data/ts-1/wal/instance
I20260805 23:12:02.699818 25971 external_mini_cluster.cc:1428] Setting key 89df092d38368a682656db17054ce331
I20260805 23:12:02.706230 5416 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.194:43897 every 8 connection(s)
WARNING: no policy specified for kudu/127.25.92.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.92.195@KRBTEST.COM" created.
Aug 05 23:12:02 dist-test-slave-ss4n krb5kdc[4537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971522, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.194@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM
I20260805 23:12:02.737752 5278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:12:02.718297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.194:33751 (local address 127.25.92.254:37969)
0805 23:12:02.718472 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0805 23:12:02.718479 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0805 23:12:02.719251 (+ 772us) server_negotiation.cc:407] Connection header received
0805 23:12:02.723378 (+ 4127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:12:02.723384 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:12:02.723476 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:12:02.723551 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:12:02.725604 (+ 2053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:12:02.726394 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:12:02.728359 (+ 1965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:12:02.728733 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:12:02.728834 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:12:02.733078 (+ 4244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:12:02.733082 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:12:02.733085 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:12:02.733116 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:12:02.735055 (+ 1939us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:12:02.735788 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:12:02.735792 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:12:02.735794 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:12:02.735892 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:12:02.736343 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:12:02.736346 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:12:02.736348 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:12:02.736586 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:12:02.736676 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0805 23:12:02.737414 (+ 738us) server_negotiation.cc:299] Negotiation successful
0805 23:12:02.737580 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260805 23:12:02.739622 5420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:12:02.716945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:37969 (local address 127.25.92.194:33751)
0805 23:12:02.719024 (+ 2079us) negotiation.cc:107] Waiting for socket to connect
0805 23:12:02.719098 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0805 23:12:02.723011 (+ 3913us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:12:02.724077 (+ 1066us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:12:02.724089 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:12:02.724526 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:12:02.725329 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:12:02.725352 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:12:02.726711 (+ 1359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:12:02.726716 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:12:02.728177 (+ 1461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:12:02.728192 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:12:02.729092 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:12:02.729097 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:12:02.729330 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:12:02.731044 (+ 1714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:12:02.731065 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:12:02.732842 (+ 1777us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:12:02.735240 (+ 2398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:12:02.735246 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:12:02.735266 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:12:02.735656 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:12:02.736009 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:12:02.736012 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:12:02.736014 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:12:02.736223 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:12:02.736749 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:12:02.736756 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:12:02.737093 (+ 337us) client_negotiation.cc:769] Sending connection context
0805 23:12:02.737937 (+ 844us) client_negotiation.cc:240] Negotiation successful
0805 23:12:02.738435 (+ 498us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1822,"thread_start_us":201,"threads_started":1}
I20260805 23:12:02.741047 5417 heartbeater.cc:347] Connected to a master server at 127.25.92.254:37969
I20260805 23:12:02.741317 5417 heartbeater.cc:464] Registering TS with master...
I20260805 23:12:02.742010 5417 heartbeater.cc:511] Master 127.25.92.254:37969 requested a full tablet report, sending...
I20260805 23:12:02.743433 5042 ts_manager.cc:194] Registered new tserver with Master: 3c6061a044a24b609169f6c25b8a9819 (127.25.92.194:43897)
I20260805 23:12:02.744194 5042 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.92.194@KRBTEST.COM'} at 127.25.92.194:33751
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.92.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.195.keytab.
Entry for principal kudu/127.25.92.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.195.keytab.
I20260805 23:12:02.753887 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.25.92.212:43995/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.92.195:0
--local_ip_for_outbound_sockets=127.25.92.195
--webserver_interface=127.25.92.195
--webserver_port=0
--tserver_master_addrs=127.25.92.254:37969
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.195.keytab
--principal=kudu/127.25.92.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:38245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:12:02.921124 5424 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:12:02.921542 5424 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:12:02.921608 5424 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:12:02.927623 5424 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:12:02.927713 5424 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:12:02.927800 5424 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.92.195
I20260805 23:12:02.936300 5424 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:38245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.25.92.212:43995/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.195.keytab
--principal=kudu/127.25.92.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.92.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.92.195
--webserver_port=0
--tserver_master_addrs=127.25.92.254:37969
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5424
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.92.195
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:12:02.937925 5424 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:12:02.939481 5424 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:12:02.953954 5430 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:12:02.954155 5429 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:12:02.954658 5432 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:12:02.956816 5424 server_base.cc:1034] running on GCE node
I20260805 23:12:02.957369 5424 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:12:02.958308 5424 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:12:02.959563 5424 hybrid_clock.cc:648] HybridClock initialized: now 1785971522959492 us; error 84 us; skew 500 ppm
Aug 05 23:12:02 dist-test-slave-ss4n krb5kdc[4537](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971522, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:12:02.963548 5424 init.cc:377] Logged in from keytab as kudu/127.25.92.195@KRBTEST.COM (short username kudu)
I20260805 23:12:02.964706 5424 webserver.cc:460] Webserver started at http://127.25.92.195:43541/ using document root <none> and password file <none>
I20260805 23:12:02.965454 5424 fs_manager.cc:359] Metadata directory not provided
I20260805 23:12:02.965574 5424 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:12:02.965840 5424 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 05 23:12:02 dist-test-slave-ss4n krb5kdc[4537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971522, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.254@KRBTEST.COM for HTTP/127.25.92.212@KRBTEST.COM
Aug 05 23:12:02 dist-test-slave-ss4n krb5kdc[4537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971522, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.195@KRBTEST.COM for HTTP/127.25.92.212@KRBTEST.COM
I20260805 23:12:02.991832 5424 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/minicluster-data/ts-2/data/instance:
uuid: "aa3c24289e1f4c49a75cc8847f378c63"
format_stamp: "Formatted at 2026-08-05 23:12:02 on dist-test-slave-ss4n"
server_key: "c9acd960e1c905d6610378cf2bc603c2"
server_key_iv: "b7d7e0897574a9b0b7d90de6eaa13504"
server_key_version: "kuduclusterkey@0"
I20260805 23:12:02.992828 5424 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/minicluster-data/ts-2/wal/instance:
uuid: "aa3c24289e1f4c49a75cc8847f378c63"
format_stamp: "Formatted at 2026-08-05 23:12:02 on dist-test-slave-ss4n"
server_key: "c9acd960e1c905d6610378cf2bc603c2"
server_key_iv: "b7d7e0897574a9b0b7d90de6eaa13504"
server_key_version: "kuduclusterkey@0"
I20260805 23:12:02.998291 5424 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260805 23:12:03.020073 5439 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:12:03.021808 5424 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.002s sys 0.000s
I20260805 23:12:03.021958 5424 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/minicluster-data/ts-2/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/minicluster-data/ts-2/wal
uuid: "aa3c24289e1f4c49a75cc8847f378c63"
format_stamp: "Formatted at 2026-08-05 23:12:02 on dist-test-slave-ss4n"
server_key: "c9acd960e1c905d6610378cf2bc603c2"
server_key_iv: "b7d7e0897574a9b0b7d90de6eaa13504"
server_key_version: "kuduclusterkey@0"
I20260805 23:12:03.022119 5424 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:12:03.049360 5424 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:12:03.053390 5424 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:12:03.053692 5424 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:12:03.054466 5424 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:12:03.055611 5424 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:12:03.055716 5424 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:12:03.055810 5424 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:12:03.055871 5424 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:12:03.068349 5424 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.195:44631
I20260805 23:12:03.069716 5424 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/minicluster-data/ts-2/data/info.pb
I20260805 23:12:03.071622 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 5424
I20260805 23:12:03.071753 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/minicluster-data/ts-2/wal/instance
I20260805 23:12:03.071909 5562 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.195:44631 every 8 connection(s)
I20260805 23:12:03.072105 25971 external_mini_cluster.cc:1428] Setting key e386f34acbe32ffc4b2952e501ec29e8
Aug 05 23:12:03 dist-test-slave-ss4n krb5kdc[4537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971522, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.195@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM
I20260805 23:12:03.096541 5278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:12:03.078704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.195:40397 (local address 127.25.92.254:37969)
0805 23:12:03.078837 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0805 23:12:03.078842 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:12:03.079414 (+ 572us) server_negotiation.cc:407] Connection header received
0805 23:12:03.080761 (+ 1347us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:12:03.080766 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:12:03.080824 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:12:03.080880 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:12:03.082727 (+ 1847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:12:03.083601 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:12:03.087046 (+ 3445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:12:03.087409 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:12:03.087494 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:12:03.091460 (+ 3966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:12:03.091465 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:12:03.091468 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:12:03.091499 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:12:03.093623 (+ 2124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:12:03.094401 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:12:03.094406 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:12:03.094409 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:12:03.094508 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:12:03.094981 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:12:03.094985 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:12:03.094988 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:12:03.095244 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:12:03.095342 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0805 23:12:03.096193 (+ 851us) server_negotiation.cc:299] Negotiation successful
0805 23:12:03.096353 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260805 23:12:03.100208 5567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:12:03.077287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:37969 (local address 127.25.92.195:40397)
0805 23:12:03.079241 (+ 1954us) negotiation.cc:107] Waiting for socket to connect
0805 23:12:03.079308 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0805 23:12:03.080495 (+ 1187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:12:03.081319 (+ 824us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:12:03.081329 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:12:03.081785 (+ 456us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:12:03.082542 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:12:03.082562 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:12:03.083778 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:12:03.083781 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:12:03.086883 (+ 3102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:12:03.086896 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:12:03.087731 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:12:03.087735 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:12:03.087952 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:12:03.088897 (+ 945us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:12:03.088935 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:12:03.091168 (+ 2233us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:12:03.093796 (+ 2628us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:12:03.093807 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:12:03.093828 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:12:03.094259 (+ 431us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:12:03.094631 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:12:03.094635 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:12:03.094637 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:12:03.094844 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:12:03.095419 (+ 575us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:12:03.095427 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:12:03.095844 (+ 417us) client_negotiation.cc:769] Sending connection context
0805 23:12:03.097768 (+ 1924us) client_negotiation.cc:240] Negotiation successful
0805 23:12:03.098902 (+ 1134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1714,"thread_start_us":216,"threads_started":1}
I20260805 23:12:03.101960 5565 heartbeater.cc:347] Connected to a master server at 127.25.92.254:37969
I20260805 23:12:03.102241 5565 heartbeater.cc:464] Registering TS with master...
I20260805 23:12:03.102854 5565 heartbeater.cc:511] Master 127.25.92.254:37969 requested a full tablet report, sending...
I20260805 23:12:03.104209 5042 ts_manager.cc:194] Registered new tserver with Master: aa3c24289e1f4c49a75cc8847f378c63 (127.25.92.195:44631)
I20260805 23:12:03.105069 5042 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.92.195@KRBTEST.COM'} at 127.25.92.195:40397
I20260805 23:12:03.110679 25971 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260805 23:12:03.132869 5278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:12:03.113993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37560 (local address 127.25.92.254:37969)
0805 23:12:03.114150 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0805 23:12:03.114155 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:12:03.122001 (+ 7846us) server_negotiation.cc:407] Connection header received
0805 23:12:03.122285 (+ 284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:12:03.122290 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:12:03.122352 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:12:03.122419 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:12:03.123652 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:12:03.124619 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:12:03.126059 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:12:03.126437 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:12:03.126527 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:12:03.128701 (+ 2174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:12:03.128705 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:12:03.128709 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:12:03.128790 (+ 81us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:12:03.130727 (+ 1937us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:12:03.131318 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:12:03.131322 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:12:03.131324 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:12:03.131449 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:12:03.131825 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:12:03.131829 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:12:03.131831 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:12:03.132076 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:12:03.132187 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0805 23:12:03.132567 (+ 380us) server_negotiation.cc:299] Negotiation successful
0805 23:12:03.132697 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260805 23:12:03.137272 5042 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37560:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260805 23:12:03.142498 5042 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260805 23:12:03.165151 5578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:12:03.154629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.193:42925 (local address 127.0.0.1:33242)
0805 23:12:03.156141 (+ 1512us) negotiation.cc:107] Waiting for socket to connect
0805 23:12:03.156200 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0805 23:12:03.156337 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:12:03.158602 (+ 2265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:12:03.158605 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:12:03.158621 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:12:03.158726 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:12:03.158733 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:12:03.161238 (+ 2505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:12:03.161242 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:12:03.163420 (+ 2178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:12:03.163431 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:12:03.164568 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:12:03.164573 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:12:03.164703 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:12:03.164754 (+ 51us) client_negotiation.cc:769] Sending connection context
0805 23:12:03.164847 (+ 93us) client_negotiation.cc:240] Negotiation successful
0805 23:12:03.164984 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1315,"thread_start_us":121,"threads_started":1}
I20260805 23:12:03.167224 5581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:12:03.155878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33242 (local address 127.25.92.193:42925)
0805 23:12:03.158144 (+ 2266us) server_negotiation.cc:206] Beginning negotiation
0805 23:12:03.158151 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0805 23:12:03.158174 (+ 23us) server_negotiation.cc:407] Connection header received
0805 23:12:03.158247 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:12:03.158252 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:12:03.158440 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:12:03.158878 (+ 438us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:12:03.159032 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:12:03.161094 (+ 2062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:12:03.163578 (+ 2484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:12:03.164424 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:12:03.165846 (+ 1422us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:12:03.166040 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0805 23:12:03.166097 (+ 57us) server_negotiation.cc:299] Negotiation successful
0805 23:12:03.166194 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2138,"thread_start_us":124,"threads_started":1}
I20260805 23:12:03.167433 5579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:12:03.155441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.194:43897 (local address 127.0.0.1:37160)
0805 23:12:03.156868 (+ 1427us) negotiation.cc:107] Waiting for socket to connect
0805 23:12:03.156882 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0805 23:12:03.156971 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:12:03.158352 (+ 1381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:12:03.158355 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:12:03.158372 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:12:03.158476 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:12:03.158486 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:12:03.159949 (+ 1463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:12:03.159953 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:12:03.162473 (+ 2520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:12:03.162487 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:12:03.166993 (+ 4506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:12:03.166997 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:12:03.167096 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:12:03.167122 (+ 26us) client_negotiation.cc:769] Sending connection context
0805 23:12:03.167200 (+ 78us) client_negotiation.cc:240] Negotiation successful
0805 23:12:03.167292 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1365,"thread_start_us":94,"threads_started":1}
I20260805 23:12:03.167970 5580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:12:03.155637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37160 (local address 127.25.92.194:43897)
0805 23:12:03.157237 (+ 1600us) server_negotiation.cc:206] Beginning negotiation
0805 23:12:03.157255 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0805 23:12:03.157281 (+ 26us) server_negotiation.cc:407] Connection header received
0805 23:12:03.157381 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:12:03.157388 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:12:03.157642 (+ 254us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:12:03.158207 (+ 565us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:12:03.158619 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:12:03.159809 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:12:03.163334 (+ 3525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:12:03.166651 (+ 3317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:12:03.166763 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:12:03.166924 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0805 23:12:03.167664 (+ 740us) server_negotiation.cc:299] Negotiation successful
0805 23:12:03.167797 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1478,"thread_start_us":74,"threads_started":1}
I20260805 23:12:03.171540 5204 tablet_service.cc:1467] Processing CreateTablet for tablet b62c5889c9f44a7ebc67326fe904dba4 (DEFAULT_TABLE table=test-table [id=34e7156cfee44604a1723aa734486529]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:12:03.174268 5340 tablet_service.cc:1467] Processing CreateTablet for tablet b62c5889c9f44a7ebc67326fe904dba4 (DEFAULT_TABLE table=test-table [id=34e7156cfee44604a1723aa734486529]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:12:03.175513 5340 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b62c5889c9f44a7ebc67326fe904dba4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:12:03.179097 5204 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b62c5889c9f44a7ebc67326fe904dba4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:12:03.187799 5577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:12:03.155052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.195:44631 (local address 127.0.0.1:37578)
0805 23:12:03.156582 (+ 1530us) negotiation.cc:107] Waiting for socket to connect
0805 23:12:03.156595 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0805 23:12:03.156681 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:12:03.157811 (+ 1130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:12:03.157817 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:12:03.157840 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:12:03.157985 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:12:03.157994 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:12:03.164282 (+ 6288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:12:03.164285 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:12:03.165558 (+ 1273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:12:03.165569 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:12:03.187287 (+ 21718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:12:03.187293 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:12:03.187427 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:12:03.187464 (+ 37us) client_negotiation.cc:769] Sending connection context
0805 23:12:03.187528 (+ 64us) client_negotiation.cc:240] Negotiation successful
0805 23:12:03.187627 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1464,"thread_start_us":82,"threads_started":1}
I20260805 23:12:03.188580 5582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:12:03.156023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37578 (local address 127.25.92.195:44631)
0805 23:12:03.157322 (+ 1299us) server_negotiation.cc:206] Beginning negotiation
0805 23:12:03.157334 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0805 23:12:03.157355 (+ 21us) server_negotiation.cc:407] Connection header received
0805 23:12:03.157423 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:12:03.157428 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:12:03.157642 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:12:03.157815 (+ 173us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:12:03.161007 (+ 3192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:12:03.163175 (+ 2168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:12:03.165832 (+ 2657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:12:03.187092 (+ 21260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:12:03.188051 (+ 959us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:12:03.188244 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0805 23:12:03.188311 (+ 67us) server_negotiation.cc:299] Negotiation successful
0805 23:12:03.188425 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1199,"thread_start_us":96,"threads_started":1}
I20260805 23:12:03.192497 5493 tablet_service.cc:1467] Processing CreateTablet for tablet b62c5889c9f44a7ebc67326fe904dba4 (DEFAULT_TABLE table=test-table [id=34e7156cfee44604a1723aa734486529]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:12:03.193714 5493 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b62c5889c9f44a7ebc67326fe904dba4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:12:03.194705 5583 tablet_bootstrap.cc:492] T b62c5889c9f44a7ebc67326fe904dba4 P 3c6061a044a24b609169f6c25b8a9819: Bootstrap starting.
I20260805 23:12:03.197996 5583 tablet_bootstrap.cc:654] T b62c5889c9f44a7ebc67326fe904dba4 P 3c6061a044a24b609169f6c25b8a9819: Neither blocks nor log segments found. Creating new log.
I20260805 23:12:03.199084 5583 log.cc:824] T b62c5889c9f44a7ebc67326fe904dba4 P 3c6061a044a24b609169f6c25b8a9819: Log is configured to *not* fsync() on all Append() calls
I20260805 23:12:03.200075 5584 tablet_bootstrap.cc:492] T b62c5889c9f44a7ebc67326fe904dba4 P a9cd57e8a78a4d4cb221c5c154b93a8e: Bootstrap starting.
I20260805 23:12:03.202809 5584 tablet_bootstrap.cc:654] T b62c5889c9f44a7ebc67326fe904dba4 P a9cd57e8a78a4d4cb221c5c154b93a8e: Neither blocks nor log segments found. Creating new log.
I20260805 23:12:03.203810 5584 log.cc:824] T b62c5889c9f44a7ebc67326fe904dba4 P a9cd57e8a78a4d4cb221c5c154b93a8e: Log is configured to *not* fsync() on all Append() calls
I20260805 23:12:03.206547 5583 tablet_bootstrap.cc:492] T b62c5889c9f44a7ebc67326fe904dba4 P 3c6061a044a24b609169f6c25b8a9819: No bootstrap required, opened a new log
I20260805 23:12:03.206787 5583 ts_tablet_manager.cc:1397] T b62c5889c9f44a7ebc67326fe904dba4 P 3c6061a044a24b609169f6c25b8a9819: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.000s
I20260805 23:12:03.211866 5583 raft_consensus.cc:348] T b62c5889c9f44a7ebc67326fe904dba4 P 3c6061a044a24b609169f6c25b8a9819 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9cd57e8a78a4d4cb221c5c154b93a8e" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 42925 } } peers { permanent_uuid: "3c6061a044a24b609169f6c25b8a9819" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 43897 } } peers { permanent_uuid: "aa3c24289e1f4c49a75cc8847f378c63" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 44631 } }
I20260805 23:12:03.212359 5583 raft_consensus.cc:374] T b62c5889c9f44a7ebc67326fe904dba4 P 3c6061a044a24b609169f6c25b8a9819 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:12:03.212543 5583 raft_consensus.cc:729] T b62c5889c9f44a7ebc67326fe904dba4 P 3c6061a044a24b609169f6c25b8a9819 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c6061a044a24b609169f6c25b8a9819, State: Initialized, Role: FOLLOWER
I20260805 23:12:03.213346 5583 consensus_queue.cc:260] T b62c5889c9f44a7ebc67326fe904dba4 P 3c6061a044a24b609169f6c25b8a9819 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9cd57e8a78a4d4cb221c5c154b93a8e" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 42925 } } peers { permanent_uuid: "3c6061a044a24b609169f6c25b8a9819" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 43897 } } peers { permanent_uuid: "aa3c24289e1f4c49a75cc8847f378c63" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 44631 } }
I20260805 23:12:03.214685 5583 ts_tablet_manager.cc:1428] T b62c5889c9f44a7ebc67326fe904dba4 P 3c6061a044a24b609169f6c25b8a9819: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20260805 23:12:03.215211 5417 heartbeater.cc:503] Master 127.25.92.254:37969 was elected leader, sending a full tablet report...
W20260805 23:12:03.216459 5418 tablet.cc:2369] T b62c5889c9f44a7ebc67326fe904dba4 P 3c6061a044a24b609169f6c25b8a9819: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260805 23:12:03.219715 5584 tablet_bootstrap.cc:492] T b62c5889c9f44a7ebc67326fe904dba4 P a9cd57e8a78a4d4cb221c5c154b93a8e: No bootstrap required, opened a new log
I20260805 23:12:03.220053 5584 ts_tablet_manager.cc:1397] T b62c5889c9f44a7ebc67326fe904dba4 P a9cd57e8a78a4d4cb221c5c154b93a8e: Time spent bootstrapping tablet: real 0.020s user 0.006s sys 0.000s
I20260805 23:12:03.225337 5584 raft_consensus.cc:348] T b62c5889c9f44a7ebc67326fe904dba4 P a9cd57e8a78a4d4cb221c5c154b93a8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9cd57e8a78a4d4cb221c5c154b93a8e" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 42925 } } peers { permanent_uuid: "3c6061a044a24b609169f6c25b8a9819" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 43897 } } peers { permanent_uuid: "aa3c24289e1f4c49a75cc8847f378c63" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 44631 } }
I20260805 23:12:03.225812 5584 raft_consensus.cc:374] T b62c5889c9f44a7ebc67326fe904dba4 P a9cd57e8a78a4d4cb221c5c154b93a8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:12:03.225916 5584 raft_consensus.cc:729] T b62c5889c9f44a7ebc67326fe904dba4 P a9cd57e8a78a4d4cb221c5c154b93a8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9cd57e8a78a4d4cb221c5c154b93a8e, State: Initialized, Role: FOLLOWER
I20260805 23:12:03.225967 5587 tablet_bootstrap.cc:492] T b62c5889c9f44a7ebc67326fe904dba4 P aa3c24289e1f4c49a75cc8847f378c63: Bootstrap starting.
I20260805 23:12:03.226645 5584 consensus_queue.cc:260] T b62c5889c9f44a7ebc67326fe904dba4 P a9cd57e8a78a4d4cb221c5c154b93a8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9cd57e8a78a4d4cb221c5c154b93a8e" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 42925 } } peers { permanent_uuid: "3c6061a044a24b609169f6c25b8a9819" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 43897 } } peers { permanent_uuid: "aa3c24289e1f4c49a75cc8847f378c63" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 44631 } }
I20260805 23:12:03.228089 5584 ts_tablet_manager.cc:1428] T b62c5889c9f44a7ebc67326fe904dba4 P a9cd57e8a78a4d4cb221c5c154b93a8e: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260805 23:12:03.228664 5274 heartbeater.cc:503] Master 127.25.92.254:37969 was elected leader, sending a full tablet report...
I20260805 23:12:03.229416 5587 tablet_bootstrap.cc:654] T b62c5889c9f44a7ebc67326fe904dba4 P aa3c24289e1f4c49a75cc8847f378c63: Neither blocks nor log segments found. Creating new log.
I20260805 23:12:03.230441 5587 log.cc:824] T b62c5889c9f44a7ebc67326fe904dba4 P aa3c24289e1f4c49a75cc8847f378c63: Log is configured to *not* fsync() on all Append() calls
I20260805 23:12:03.235682 5587 tablet_bootstrap.cc:492] T b62c5889c9f44a7ebc67326fe904dba4 P aa3c24289e1f4c49a75cc8847f378c63: No bootstrap required, opened a new log
I20260805 23:12:03.235997 5587 ts_tablet_manager.cc:1397] T b62c5889c9f44a7ebc67326fe904dba4 P aa3c24289e1f4c49a75cc8847f378c63: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260805 23:12:03.242372 5587 raft_consensus.cc:348] T b62c5889c9f44a7ebc67326fe904dba4 P aa3c24289e1f4c49a75cc8847f378c63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9cd57e8a78a4d4cb221c5c154b93a8e" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 42925 } } peers { permanent_uuid: "3c6061a044a24b609169f6c25b8a9819" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 43897 } } peers { permanent_uuid: "aa3c24289e1f4c49a75cc8847f378c63" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 44631 } }
I20260805 23:12:03.242852 5587 raft_consensus.cc:374] T b62c5889c9f44a7ebc67326fe904dba4 P aa3c24289e1f4c49a75cc8847f378c63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:12:03.242975 5587 raft_consensus.cc:729] T b62c5889c9f44a7ebc67326fe904dba4 P aa3c24289e1f4c49a75cc8847f378c63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa3c24289e1f4c49a75cc8847f378c63, State: Initialized, Role: FOLLOWER
I20260805 23:12:03.243675 5587 consensus_queue.cc:260] T b62c5889c9f44a7ebc67326fe904dba4 P aa3c24289e1f4c49a75cc8847f378c63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9cd57e8a78a4d4cb221c5c154b93a8e" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 42925 } } peers { permanent_uuid: "3c6061a044a24b609169f6c25b8a9819" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 43897 } } peers { permanent_uuid: "aa3c24289e1f4c49a75cc8847f378c63" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 44631 } }
I20260805 23:12:03.245067 5587 ts_tablet_manager.cc:1428] T b62c5889c9f44a7ebc67326fe904dba4 P aa3c24289e1f4c49a75cc8847f378c63: Time spent starting tablet: real 0.009s user 0.007s sys 0.000s
I20260805 23:12:03.245585 5565 heartbeater.cc:503] Master 127.25.92.254:37969 was elected leader, sending a full tablet report...
W20260805 23:12:03.320545 5275 tablet.cc:2369] T b62c5889c9f44a7ebc67326fe904dba4 P a9cd57e8a78a4d4cb221c5c154b93a8e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260805 23:12:03.326128 5566 tablet.cc:2369] T b62c5889c9f44a7ebc67326fe904dba4 P aa3c24289e1f4c49a75cc8847f378c63: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 05 23:12:03 dist-test-slave-ss4n krb5kdc[4537](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971522, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.254@KRBTEST.COM for HTTP/127.25.92.212@KRBTEST.COM
I20260805 23:12:03.487432 5592 raft_consensus.cc:482] T b62c5889c9f44a7ebc67326fe904dba4 P aa3c24289e1f4c49a75cc8847f378c63 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 23:12:03.487705 5592 raft_consensus.cc:504] T b62c5889c9f44a7ebc67326fe904dba4 P aa3c24289e1f4c49a75cc8847f378c63 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9cd57e8a78a4d4cb221c5c154b93a8e" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 42925 } } peers { permanent_uuid: "3c6061a044a24b609169f6c25b8a9819" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 43897 } } peers { permanent_uuid: "aa3c24289e1f4c49a75cc8847f378c63" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 44631 } }
I20260805 23:12:03.489181 5592 leader_election.cc:288] T b62c5889c9f44a7ebc67326fe904dba4 P aa3c24289e1f4c49a75cc8847f378c63 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a9cd57e8a78a4d4cb221c5c154b93a8e (127.25.92.193:42925), 3c6061a044a24b609169f6c25b8a9819 (127.25.92.194:43897)
I20260805 23:12:03.494745 5567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:12:03.489649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.193:42925 (local address 127.25.92.195:53659)
0805 23:12:03.490292 (+ 643us) negotiation.cc:107] Waiting for socket to connect
0805 23:12:03.490305 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0805 23:12:03.490436 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:12:03.490800 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:12:03.490803 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:12:03.490822 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:12:03.490943 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:12:03.490951 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:12:03.492042 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:12:03.492046 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:12:03.493279 (+ 1233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:12:03.493290 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:12:03.494307 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:12:03.494311 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:12:03.494410 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:12:03.494432 (+ 22us) client_negotiation.cc:769] Sending connection context
0805 23:12:03.494510 (+ 78us) client_negotiation.cc:240] Negotiation successful
0805 23:12:03.494586 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":562}
I20260805 23:12:03.495806 5581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:12:03.489717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.195:53659 (local address 127.25.92.193:42925)
0805 23:12:03.489885 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0805 23:12:03.489890 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:12:03.490376 (+ 486us) server_negotiation.cc:407] Connection header received
0805 23:12:03.490536 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:12:03.490539 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:12:03.490587 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:12:03.490669 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:12:03.491085 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:12:03.491901 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:12:03.493427 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:12:03.494059 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:12:03.494182 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:12:03.494242 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0805 23:12:03.495580 (+ 1338us) server_negotiation.cc:299] Negotiation successful
0805 23:12:03.495682 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260805 23:12:03.496438 5215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b62c5889c9f44a7ebc67326fe904dba4" candidate_uuid: "aa3c24289e1f4c49a75cc8847f378c63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9cd57e8a78a4d4cb221c5c154b93a8e" is_pre_election: true
I20260805 23:12:03.496836 5215 raft_consensus.cc:2454] T b62c5889c9f44a7ebc67326fe904dba4 P a9cd57e8a78a4d4cb221c5c154b93a8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa3c24289e1f4c49a75cc8847f378c63 in term 0.
I20260805 23:12:03.497548 5441 leader_election.cc:302] T b62c5889c9f44a7ebc67326fe904dba4 P aa3c24289e1f4c49a75cc8847f378c63 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a9cd57e8a78a4d4cb221c5c154b93a8e, aa3c24289e1f4c49a75cc8847f378c63; no voters:
I20260805 23:12:03.497886 5592 raft_consensus.cc:2781] T b62c5889c9f44a7ebc67326fe904dba4 P aa3c24289e1f4c49a75cc8847f378c63 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260805 23:12:03.497965 5592 raft_consensus.cc:482] T b62c5889c9f44a7ebc67326fe904dba4 P aa3c24289e1f4c49a75cc8847f378c63 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260805 23:12:03.498008 5592 raft_consensus.cc:3037] T b62c5889c9f44a7ebc67326fe904dba4 P aa3c24289e1f4c49a75cc8847f378c63 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:12:03.499498 5592 raft_consensus.cc:504] T b62c5889c9f44a7ebc67326fe904dba4 P aa3c24289e1f4c49a75cc8847f378c63 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9cd57e8a78a4d4cb221c5c154b93a8e" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 42925 } } peers { permanent_uuid: "3c6061a044a24b609169f6c25b8a9819" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 43897 } } peers { permanent_uuid: "aa3c24289e1f4c49a75cc8847f378c63" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 44631 } }
I20260805 23:12:03.499891 5592 leader_election.cc:288] T b62c5889c9f44a7ebc67326fe904dba4 P aa3c24289e1f4c49a75cc8847f378c63 [CANDIDATE]: Term 1 election: Requested vote from peers a9cd57e8a78a4d4cb221c5c154b93a8e (127.25.92.193:42925), 3c6061a044a24b609169f6c25b8a9819 (127.25.92.194:43897)
I20260805 23:12:03.501261 5215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b62c5889c9f44a7ebc67326fe904dba4" candidate_uuid: "aa3c24289e1f4c49a75cc8847f378c63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9cd57e8a78a4d4cb221c5c154b93a8e"
I20260805 23:12:03.501399 5215 raft_consensus.cc:3037] T b62c5889c9f44a7ebc67326fe904dba4 P a9cd57e8a78a4d4cb221c5c154b93a8e [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:12:03.502768 5215 raft_consensus.cc:2454] T b62c5889c9f44a7ebc67326fe904dba4 P a9cd57e8a78a4d4cb221c5c154b93a8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa3c24289e1f4c49a75cc8847f378c63 in term 1.
I20260805 23:12:03.503211 5441 leader_election.cc:302] T b62c5889c9f44a7ebc67326fe904dba4 P aa3c24289e1f4c49a75cc8847f378c63 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a9cd57e8a78a4d4cb221c5c154b93a8e, aa3c24289e1f4c49a75cc8847f378c63; no voters:
I20260805 23:12:03.503441 5592 raft_consensus.cc:2781] T b62c5889c9f44a7ebc67326fe904dba4 P aa3c24289e1f4c49a75cc8847f378c63 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:12:03.503677 5592 raft_consensus.cc:686] T b62c5889c9f44a7ebc67326fe904dba4 P aa3c24289e1f4c49a75cc8847f378c63 [term 1 LEADER]: Becoming Leader. State: Replica: aa3c24289e1f4c49a75cc8847f378c63, State: Running, Role: LEADER
I20260805 23:12:03.504060 5592 consensus_queue.cc:237] T b62c5889c9f44a7ebc67326fe904dba4 P aa3c24289e1f4c49a75cc8847f378c63 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9cd57e8a78a4d4cb221c5c154b93a8e" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 42925 } } peers { permanent_uuid: "3c6061a044a24b609169f6c25b8a9819" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 43897 } } peers { permanent_uuid: "aa3c24289e1f4c49a75cc8847f378c63" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 44631 } }
I20260805 23:12:03.510457 5580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:12:03.490058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.195:33459 (local address 127.25.92.194:43897)
0805 23:12:03.490223 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0805 23:12:03.490229 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:12:03.495255 (+ 5026us) server_negotiation.cc:407] Connection header received
0805 23:12:03.495323 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:12:03.495326 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:12:03.495379 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:12:03.495456 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:12:03.506008 (+ 10552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:12:03.506814 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:12:03.508808 (+ 1994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:12:03.509375 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:12:03.509516 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:12:03.509577 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0805 23:12:03.510259 (+ 682us) server_negotiation.cc:299] Negotiation successful
0805 23:12:03.510341 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260805 23:12:03.512521 5042 catalog_manager.cc:5495] T b62c5889c9f44a7ebc67326fe904dba4 P aa3c24289e1f4c49a75cc8847f378c63 reported cstate change: term changed from 0 to 1, leader changed from <none> to aa3c24289e1f4c49a75cc8847f378c63 (127.25.92.195). New cstate: current_term: 1 leader_uuid: "aa3c24289e1f4c49a75cc8847f378c63" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9cd57e8a78a4d4cb221c5c154b93a8e" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 42925 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c6061a044a24b609169f6c25b8a9819" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 43897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa3c24289e1f4c49a75cc8847f378c63" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 44631 } health_report { overall_health: HEALTHY } } }
I20260805 23:12:03.510144 5567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:12:03.489934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.194:43897 (local address 127.25.92.195:33459)
0805 23:12:03.494935 (+ 5001us) negotiation.cc:107] Waiting for socket to connect
0805 23:12:03.494950 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0805 23:12:03.495039 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:12:03.505697 (+ 10658us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:12:03.505701 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:12:03.505729 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:12:03.505834 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:12:03.505841 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:12:03.506964 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:12:03.506968 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:12:03.508110 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:12:03.508123 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:12:03.509691 (+ 1568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:12:03.509695 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:12:03.509790 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:12:03.509814 (+ 24us) client_negotiation.cc:769] Sending connection context
0805 23:12:03.509889 (+ 75us) client_negotiation.cc:240] Negotiation successful
0805 23:12:03.509971 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4940,"thread_start_us":134,"threads_started":1}
I20260805 23:12:03.514751 5366 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b62c5889c9f44a7ebc67326fe904dba4" candidate_uuid: "aa3c24289e1f4c49a75cc8847f378c63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c6061a044a24b609169f6c25b8a9819" is_pre_election: true
I20260805 23:12:03.515105 5366 raft_consensus.cc:2454] T b62c5889c9f44a7ebc67326fe904dba4 P 3c6061a044a24b609169f6c25b8a9819 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa3c24289e1f4c49a75cc8847f378c63 in term 0.
I20260805 23:12:03.515897 5367 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b62c5889c9f44a7ebc67326fe904dba4" candidate_uuid: "aa3c24289e1f4c49a75cc8847f378c63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c6061a044a24b609169f6c25b8a9819"
I20260805 23:12:03.516054 5367 raft_consensus.cc:3037] T b62c5889c9f44a7ebc67326fe904dba4 P 3c6061a044a24b609169f6c25b8a9819 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:12:03.517627 5367 raft_consensus.cc:2454] T b62c5889c9f44a7ebc67326fe904dba4 P 3c6061a044a24b609169f6c25b8a9819 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa3c24289e1f4c49a75cc8847f378c63 in term 1.
I20260805 23:12:03.540607 5582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:12:03.534874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37582 (local address 127.25.92.195:44631)
0805 23:12:03.535281 (+ 407us) server_negotiation.cc:206] Beginning negotiation
0805 23:12:03.535286 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:12:03.535301 (+ 15us) server_negotiation.cc:407] Connection header received
0805 23:12:03.535397 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:12:03.535400 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:12:03.535486 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:12:03.535625 (+ 139us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 23:12:03.536099 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:12:03.536990 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:12:03.538465 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:12:03.538813 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:12:03.538930 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:12:03.539450 (+ 520us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 23:12:03.539999 (+ 549us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 23:12:03.540108 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0805 23:12:03.540319 (+ 211us) server_negotiation.cc:299] Negotiation successful
0805 23:12:03.540391 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314}
I20260805 23:12:03.557168 5367 raft_consensus.cc:1261] T b62c5889c9f44a7ebc67326fe904dba4 P 3c6061a044a24b609169f6c25b8a9819 [term 1 FOLLOWER]: Refusing update from remote peer aa3c24289e1f4c49a75cc8847f378c63: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:12:03.558737 5215 raft_consensus.cc:1261] T b62c5889c9f44a7ebc67326fe904dba4 P a9cd57e8a78a4d4cb221c5c154b93a8e [term 1 FOLLOWER]: Refusing update from remote peer aa3c24289e1f4c49a75cc8847f378c63: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:12:03.559777 5592 consensus_queue.cc:1035] T b62c5889c9f44a7ebc67326fe904dba4 P aa3c24289e1f4c49a75cc8847f378c63 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c6061a044a24b609169f6c25b8a9819" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 43897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:12:03.560251 5592 consensus_queue.cc:1035] T b62c5889c9f44a7ebc67326fe904dba4 P aa3c24289e1f4c49a75cc8847f378c63 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9cd57e8a78a4d4cb221c5c154b93a8e" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 42925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:12:03.596921 5602 mvcc.cc:205] Tried to move back new op lower bound from 7315339360480645120 to 7315339360275914752. Current Snapshot: MvccSnapshot[applied={T|T < 7315339360480645120}]
I20260805 23:12:03.597220 5600 mvcc.cc:205] Tried to move back new op lower bound from 7315339360480645120 to 7315339360275914752. Current Snapshot: MvccSnapshot[applied={T|T < 7315339360480645120}]
I20260805 23:12:03.605826 5042 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37560:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:12:03.606055 5042 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37560:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:12:03.609699 5042 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 66baab3ba0284421a220eed88f7089da: Sending DeleteTablet for 3 replicas of tablet b62c5889c9f44a7ebc67326fe904dba4
I20260805 23:12:03.610857 5493 tablet_service.cc:1514] Processing DeleteTablet for tablet b62c5889c9f44a7ebc67326fe904dba4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:12:03 UTC) from {username='kudu', principal='kudu/127.25.92.254@KRBTEST.COM'} at 127.0.0.1:37578
I20260805 23:12:03.612203 5204 tablet_service.cc:1514] Processing DeleteTablet for tablet b62c5889c9f44a7ebc67326fe904dba4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:12:03 UTC) from {username='kudu', principal='kudu/127.25.92.254@KRBTEST.COM'} at 127.0.0.1:33242
I20260805 23:12:03.612789 5340 tablet_service.cc:1514] Processing DeleteTablet for tablet b62c5889c9f44a7ebc67326fe904dba4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:12:03 UTC) from {username='kudu', principal='kudu/127.25.92.254@KRBTEST.COM'} at 127.0.0.1:37160
I20260805 23:12:03.614298 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 5130
I20260805 23:12:03.621727 5609 tablet_replica.cc:330] T b62c5889c9f44a7ebc67326fe904dba4 P aa3c24289e1f4c49a75cc8847f378c63: stopping tablet replica
I20260805 23:12:03.622144 5609 raft_consensus.cc:2229] T b62c5889c9f44a7ebc67326fe904dba4 P aa3c24289e1f4c49a75cc8847f378c63 [term 1 LEADER]: Raft consensus shutting down.
I20260805 23:12:03.622673 5609 raft_consensus.cc:2258] T b62c5889c9f44a7ebc67326fe904dba4 P aa3c24289e1f4c49a75cc8847f378c63 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:12:03.624423 5609 ts_tablet_manager.cc:1905] T b62c5889c9f44a7ebc67326fe904dba4 P aa3c24289e1f4c49a75cc8847f378c63: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:12:03.624960 5612 tablet_replica.cc:330] T b62c5889c9f44a7ebc67326fe904dba4 P 3c6061a044a24b609169f6c25b8a9819: stopping tablet replica
I20260805 23:12:03.625394 5612 raft_consensus.cc:2229] T b62c5889c9f44a7ebc67326fe904dba4 P 3c6061a044a24b609169f6c25b8a9819 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 23:12:03.625896 5612 raft_consensus.cc:2258] T b62c5889c9f44a7ebc67326fe904dba4 P 3c6061a044a24b609169f6c25b8a9819 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:12:03.627741 5609 ts_tablet_manager.cc:1918] T b62c5889c9f44a7ebc67326fe904dba4 P aa3c24289e1f4c49a75cc8847f378c63: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 23:12:03.627846 5609 log.cc:1196] T b62c5889c9f44a7ebc67326fe904dba4 P aa3c24289e1f4c49a75cc8847f378c63: Deleting WAL directory at /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971406251336-25971-0/minicluster-data/ts-2/wal/wals/b62c5889c9f44a7ebc67326fe904dba4
I20260805 23:12:03.628171 5609 ts_tablet_manager.cc:1939] T b62c5889c9f44a7ebc67326fe904dba4 P aa3c24289e1f4c49a75cc8847f378c63: Deleting consensus metadata
I20260805 23:12:03.629364 5612 ts_tablet_manager.cc:1905] T b62c5889c9f44a7ebc67326fe904dba4 P 3c6061a044a24b609169f6c25b8a9819: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:12:03.630620 5031 catalog_manager.cc:4841] TS aa3c24289e1f4c49a75cc8847f378c63 (127.25.92.195:44631): tablet b62c5889c9f44a7ebc67326fe904dba4 (table test-table [id=34e7156cfee44604a1723aa734486529]) successfully deleted
W20260805 23:12:03.635301 5030 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.25.92.193:42925 (error 108)
I20260805 23:12:03.636703 5612 ts_tablet_manager.cc:1918] T b62c5889c9f44a7ebc67326fe904dba4 P 3c6061a044a24b609169f6c25b8a9819: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 23:12:03.637024 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 5282
I20260805 23:12:03.643332 5577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:12:03.642841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.193:42925 (local address 127.0.0.1:33244)
0805 23:12:03.643126 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0805 23:12:03.643221 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.193:42925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174}
W20260805 23:12:03.643703 5030 catalog_manager.cc:4584] TS a9cd57e8a78a4d4cb221c5c154b93a8e (127.25.92.193:42925): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b62c5889c9f44a7ebc67326fe904dba4: Network error: Client connection negotiation failed: client connection to 127.25.92.193:42925: connect: Connection refused (error 111)
W20260805 23:12:03.649380 5577 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:12:03.648682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.194:43897 (local address 127.0.0.1:37164)
0805 23:12:03.648886 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0805 23:12:03.648902 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0805 23:12:03.649020 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:12:03.649259 (+ 239us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.194:43897: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":124}
I20260805 23:12:03.650219 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 5424
I20260805 23:12:03.657357 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 5011
I20260805 23:12:03.667918 25971 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260805 23:12:04.013787 25971 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260805 23:12:04.013919 25971 mini_ranger.cc:66] Stopping Ranger...
I20260805 23:12:04.359858 25971 mini_ranger.cc:68] Stopped Ranger
2026-08-05 23:12:04.360 UTC [4569] LOG: received smart shutdown request
2026-08-05 23:12:04.364 UTC [4569] LOG: background worker "logical replication launcher" (PID 4578) exited with exit code 1
2026-08-05 23:12:04.365 UTC [4573] LOG: shutting down
2026-08-05 23:12:04.580 UTC [4569] LOG: database system is shut down
2026-08-05T23:12:04Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (51988 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 23:12:04 dist-test-slave-ss4n krb5kdc[5623](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 23:12:04 dist-test-slave-ss4n krb5kdc[5623](info): set up 2 sockets
Aug 05 23:12:04 dist-test-slave-ss4n krb5kdc[5623](info): commencing operation
krb5kdc: starting...
W20260805 23:12:05.807750 25971 mini_kdc.cc:121] Time spent starting KDC: real 1.076s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/krb5kdc/test-user.keytab.
Aug 05 23:12:05 dist-test-slave-ss4n krb5kdc[5623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971525, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T23:12:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:12:05Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.25.92.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.25.92.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.25.92.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/krb5kdc/rangeradmin_127.25.92.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.25.92.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/krb5kdc/rangeradmin_127.25.92.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.25.92.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.25.92.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.25.92.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/krb5kdc/rangerlookup_127.25.92.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.25.92.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/krb5kdc/rangerlookup_127.25.92.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.25.92.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.25.92.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.25.92.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/krb5kdc/HTTP_127.25.92.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.25.92.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/krb5kdc/HTTP_127.25.92.212@KRBTEST.COM.keytab.
I20260805 23:12:06.000540 25971 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskd06taO/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/postgres -l logfile start
2026-08-05 23:12:08.804 UTC [5655] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-05 23:12:08.805 UTC [5655] LOG: listening on IPv4 address "127.25.92.212", port 46629
2026-08-05 23:12:08.809 UTC [5655] LOG: listening on Unix socket "/tmp/.s.PGSQL.46629"
2026-08-05 23:12:08.834 UTC [5658] LOG: database system was shut down at 2026-08-05 23:12:07 UTC
2026-08-05 23:12:08.838 UTC [5655] LOG: database system is ready to accept connections
I20260805 23:12:09.859732 25971 mini_postgres.cc:96] Postgres bound to 46629
2026-08-05 23:12:09.866 UTC [5666] FATAL: database "slave" does not exist
127.25.92.212:46629 - accepting connections
I20260805 23:12:09.867245 25971 mini_ranger.cc:161] Starting Ranger...
I20260805 23:12:09.885228 25971 mini_ranger.cc:84] Created miniranger Postgres user
I20260805 23:12:10.448223 25971 mini_ranger.cc:87] Created ranger Postgres database
I20260805 23:12:10.448328 25971 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ranger-admin
2026-08-05 23:12:10,736 [I] DB FLAVOR :POSTGRES
2026-08-05 23:12:10,737 [I] --------- Verifying Ranger DB connection ---------
2026-08-05 23:12:10,737 [I] Checking connection..
2026-08-05 23:12:10,737 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskd06taO/build/debug/bin/postgresql.jar:/tmp/dist-test-taskd06taO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.92.212:46629/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-05 23:12:11,018 [I] Checking connection passed.
2026-08-05 23:12:11,018 [I] --------- Verifying version history table ---------
2026-08-05 23:12:11,018 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskd06taO/build/debug/bin/postgresql.jar:/tmp/dist-test-taskd06taO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.92.212:46629/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-05 23:12:11,291 [I] Table x_db_version_h does not exist in database ranger
2026-08-05 23:12:11,292 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskd06taO/build/debug/bin/postgresql.jar:/tmp/dist-test-taskd06taO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.92.212:46629/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-05 23:12:11,543 [I] Table x_db_version_h does not exist in database ranger
2026-08-05 23:12:11,543 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-05 23:12:11,543 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskd06taO/build/debug/bin/postgresql.jar:/tmp/dist-test-taskd06taO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.92.212:46629/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskd06taO/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-05 23:12:11.798 UTC [5732] WARNING: there is no transaction in progress
2026-08-05 23:12:11,805 [I] create_dbversion_catalog.sql file imported successfully
2026-08-05 23:12:11,805 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskd06taO/build/debug/bin/postgresql.jar:/tmp/dist-test-taskd06taO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.92.212:46629/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-05 23:12:12,054 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-05 23:12:12,055 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-05 23:12:12,055 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskd06taO/build/debug/bin/postgresql.jar:/tmp/dist-test-taskd06taO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.92.212:46629/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-05 23:12:12,301 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskd06taO/build/debug/bin/postgresql.jar:/tmp/dist-test-taskd06taO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.92.212:46629/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-05 23:12:12,553 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskd06taO/build/debug/bin/postgresql.jar:/tmp/dist-test-taskd06taO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.92.212:46629/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-ss4n.c.gcp-upstream.internal','N') ;"
2026-08-05 23:12:12,829 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskd06taO/build/debug/bin/postgresql.jar:/tmp/dist-test-taskd06taO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.92.212:46629/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-05 23:12:13,061 [I] Table x_portal_user does not exist in database ranger
2026-08-05 23:12:13,061 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskd06taO/build/debug/bin/postgresql.jar:/tmp/dist-test-taskd06taO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.92.212:46629/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-05 23:12:13,324 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-05 23:12:13,324 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskd06taO/build/debug/bin/postgresql.jar:/tmp/dist-test-taskd06taO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.92.212:46629/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-05 23:12:13,563 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskd06taO/build/debug/bin/postgresql.jar:/tmp/dist-test-taskd06taO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.92.212:46629/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-05 23:12:13,824 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-05 23:12:13,824 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskd06taO/build/debug/bin/postgresql.jar:/tmp/dist-test-taskd06taO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.92.212:46629/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskd06taO/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-05 23:12:14.956 UTC [5853] WARNING: there is no transaction in progress
2026-08-05 23:12:14.975 UTC [5853] WARNING: there is no transaction in progress
2026-08-05 23:12:14.996 UTC [5853] WARNING: there is no transaction in progress
2026-08-05 23:12:15.015 UTC [5853] WARNING: there is no transaction in progress
2026-08-05 23:12:15.035 UTC [5853] WARNING: there is no transaction in progress
2026-08-05 23:12:15.059 UTC [5853] WARNING: there is no transaction in progress
2026-08-05 23:12:15.120 UTC [5853] WARNING: there is no transaction in progress
2026-08-05 23:12:15.128 UTC [5853] WARNING: there is no transaction in progress
2026-08-05 23:12:15.146 UTC [5853] WARNING: there is no transaction in progress
2026-08-05 23:12:15.158 UTC [5853] WARNING: there is no transaction in progress
2026-08-05 23:12:15.171 UTC [5853] WARNING: there is no transaction in progress
2026-08-05 23:12:15.186 UTC [5853] WARNING: there is no transaction in progress
2026-08-05 23:12:15.198 UTC [5853] WARNING: there is no transaction in progress
2026-08-05 23:12:15.207 UTC [5853] WARNING: there is no transaction in progress
2026-08-05 23:12:15.215 UTC [5853] WARNING: there is no transaction in progress
2026-08-05 23:12:15.223 UTC [5853] WARNING: there is no transaction in progress
2026-08-05 23:12:16,044 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-05 23:12:16,044 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskd06taO/build/debug/bin/postgresql.jar:/tmp/dist-test-taskd06taO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.92.212:46629/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-05 23:12:16,286 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-05 23:12:16,286 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskd06taO/build/debug/bin/postgresql.jar:/tmp/dist-test-taskd06taO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.92.212:46629/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-05 23:12:16,567 [I] Table x_portal_user already exists in database 'ranger'
2026-08-05 23:12:16,568 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskd06taO/build/debug/bin/postgresql.jar:/tmp/dist-test-taskd06taO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.92.212:46629/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-05 23:12:16,824 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-05 23:12:16,824 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskd06taO/build/debug/bin/postgresql.jar:/tmp/dist-test-taskd06taO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.92.212:46629/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-05 23:12:17,089 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskd06taO/build/debug/bin/postgresql.jar:/tmp/dist-test-taskd06taO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.92.212:46629/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-05 23:12:17,347 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskd06taO/build/debug/bin/postgresql.jar:/tmp/dist-test-taskd06taO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.92.212:46629/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-ss4n.c.gcp-upstream.internal';"
2026-08-05 23:12:17,600 [I] CORE_DB_SCHEMA import status has been updated
2026-08-05 23:12:17,601 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskd06taO/build/debug/bin/postgresql.jar:/tmp/dist-test-taskd06taO/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.92.212:46629/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-05 23:12:17,846 [I] DB_PATCHES have already been applied
I20260805 23:12:17.852196 25971 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ranger-admin:/tmp/dist-test-taskd06taO/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskd06taO/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskd06taO/build/debug/bin/postgresql.jar:/tmp/dist-test-taskd06taO/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260805 23:12:17.852264 25971 mini_ranger.cc:193] Using host: 127.25.92.212
I20260805 23:12:17.856194 25971 mini_ranger.cc:227] Ranger admin URL: http://127.25.92.212:36029
Aug 05, 2026 11:12:18 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 05, 2026 11:12:18 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 05, 2026 11:12:18 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ranger-admin/ews/webapp
Aug 05, 2026 11:12:18 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 05, 2026 11:12:18 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 05, 2026 11:12:18 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ranger-admin/ews/webapp].
Aug 05, 2026 11:12:18 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260805 23:12:19.049916 25971 mini_ranger.cc:160] Time spent starting Ranger: real 9.183s user 0.000s sys 0.011s
Aug 05, 2026 11:12:41 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 05, 2026 11:12:41 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 05, 2026 11:12:41 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 05, 2026 11:12:41 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 05, 2026 11:12:41 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 05, 2026 11:12:41 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 05, 2026 11:12:41 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 05, 2026 11:12:41 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 05, 2026 11:12:41 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 05, 2026 11:12:41 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 05, 2026 11:12:41 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 05, 2026 11:12:41 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 05, 2026 11:12:41 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 05, 2026 11:12:41 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 05, 2026 11:12:41 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 05, 2026 11:12:41 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 05, 2026 11:12:41 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 05, 2026 11:12:42 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260805 23:12:43.098129 25971 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.25.92.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.25.92.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.25.92.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/krb5kdc/rangerkms_127.25.92.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.25.92.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/krb5kdc/rangerkms_127.25.92.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.25.92.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.25.92.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.25.92.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/krb5kdc/HTTP_127.25.92.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.25.92.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/krb5kdc/HTTP_127.25.92.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 05 23:12:43 dist-test-slave-ss4n krb5kdc[5623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971563, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260805 23:12:43.200834 25971 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260805 23:12:43.212373 25971 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260805 23:12:44.307231 25971 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260805 23:12:44.307324 25971 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ranger-kms
I20260805 23:12:44.307348 25971 mini_ranger_kms.cc:227] Using postgres at 127.25.92.212:46629
2026-08-05 23:12:44,511 [I] DB FLAVOR :POSTGRES
2026-08-05 23:12:44,511 [I] --------- Verifying Ranger DB connection ---------
2026-08-05 23:12:44,511 [I] Checking connection
2026-08-05 23:12:44,511 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskd06taO/build/debug/bin/postgresql.jar:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.92.212:46629/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-05 23:12:44,833 [I] connection success
2026-08-05 23:12:44,833 [I] --------- Verifying Ranger DB tables ---------
2026-08-05 23:12:44,833 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-05 23:12:44,833 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskd06taO/build/debug/bin/postgresql.jar:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.92.212:46629/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-05 23:12:45,115 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-05 23:12:45,116 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-05 23:12:45,116 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-05 23:12:45,116 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskd06taO/build/debug/bin/postgresql.jar:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.92.212:46629/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-05 23:12:45,397 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260805 23:12:45.908396 25971 mini_ranger_kms.cc:310] Created kms service
I20260805 23:12:46.115034 25971 mini_ranger_kms.cc:326] Created kudu user
I20260805 23:12:46.170594 25971 mini_ranger_kms.cc:343] Created rangerkms user
Aug 05 23:12:46 dist-test-slave-ss4n krb5kdc[5623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971563, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.25.92.212@KRBTEST.COM
I20260805 23:12:46.613399 25971 mini_ranger_kms.cc:384] Added ranger policy
I20260805 23:12:46.613600 25971 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ranger-kms:/tmp/dist-test-taskd06taO/build/debug/bin/postgresql.jar:/tmp/dist-test-taskd06taO/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskd06taO/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260805 23:12:46.613641 25971 mini_ranger_kms.cc:240] Using host: 127.25.92.212
I20260805 23:12:46.619099 25971 mini_ranger_kms.cc:276] Ranger KMS PID: 6045
I20260805 23:12:46.619201 25971 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.25.92.212:46029
Aug 05, 2026 11:12:47 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 05, 2026 11:12:47 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 05, 2026 11:12:47 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 05, 2026 11:12:47 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 05, 2026 11:12:48 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ranger-kms/ews/webapp].
Aug 05, 2026 11:12:48 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 05 23:12:48 dist-test-slave-ss4n krb5kdc[5623](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1785971568, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.25.92.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:12:48.953931 25971 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.753s user 0.004s sys 0.173s
I20260805 23:12:48.954078 25971 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260805 23:12:48.954177 25971 mini_ranger_kms.cc:401] 127.25.92.212:46029/kms/v1/keys
Aug 05 23:12:49 dist-test-slave-ss4n krb5kdc[5623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971568, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.25.92.212@KRBTEST.COM for HTTP/127.25.92.212@KRBTEST.COM
Aug 05 23:12:49 dist-test-slave-ss4n krb5kdc[5623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971568, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.25.92.212@KRBTEST.COM for HTTP/127.25.92.212@KRBTEST.COM
Aug 05 23:12:52 dist-test-slave-ss4n krb5kdc[5623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971572, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.25.92.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.92.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.92.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab.
Entry for principal kudu/127.25.92.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab.
I20260805 23:12:52.090428 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.254:44315
--webserver_interface=127.25.92.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab
--principal=kudu/127.25.92.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:35971
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.92.254:44315
--ranger_config_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.25.92.212:46029/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:12:52.215462 6096 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:12:52.215785 6096 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:12:52.215826 6096 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:12:52.220217 6096 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:12:52.220340 6096 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:12:52.220376 6096 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:12:52.220422 6096 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:12:52.220456 6096 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:12:52.226465 6096 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:35971
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/master-0/wal
--ranger_kms_url=127.25.92.212:46029/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.25.92.254:44315
--ranger_config_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab
--principal=kudu/127.25.92.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.254:44315
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.92.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6096
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:12:52.227747 6096 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:12:52.228822 6096 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:12:52.238327 6104 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:12:52.238359 6101 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:12:52.238816 6096 server_base.cc:1034] running on GCE node
W20260805 23:12:52.238818 6102 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:12:52.239629 6096 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:12:52.240896 6096 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:12:52.242138 6096 hybrid_clock.cc:648] HybridClock initialized: now 1785971572242104 us; error 54 us; skew 500 ppm
Aug 05 23:12:52 dist-test-slave-ss4n krb5kdc[5623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971572, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:12:52.250900 6096 init.cc:377] Logged in from keytab as kudu/127.25.92.254@KRBTEST.COM (short username kudu)
I20260805 23:12:52.252166 6096 webserver.cc:460] Webserver started at http://127.25.92.254:38815/ using document root <none> and password file <none>
I20260805 23:12:52.252759 6096 fs_manager.cc:359] Metadata directory not provided
I20260805 23:12:52.252846 6096 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:12:52.253085 6096 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 05 23:12:52 dist-test-slave-ss4n krb5kdc[5623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971572, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.254@KRBTEST.COM for HTTP/127.25.92.212@KRBTEST.COM
I20260805 23:12:52.303471 6096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/master-0/data/instance:
uuid: "ba0742b645c94c13b3db716954ead8aa"
format_stamp: "Formatted at 2026-08-05 23:12:52 on dist-test-slave-ss4n"
server_key: "9b5e195756cfd7ca7f9f52451d7af6f6"
server_key_iv: "ab06c2ff8a4468c0670b87c8179a0d21"
server_key_version: "kuduclusterkey@0"
I20260805 23:12:52.304175 6096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/master-0/wal/instance:
uuid: "ba0742b645c94c13b3db716954ead8aa"
format_stamp: "Formatted at 2026-08-05 23:12:52 on dist-test-slave-ss4n"
server_key: "9b5e195756cfd7ca7f9f52451d7af6f6"
server_key_iv: "ab06c2ff8a4468c0670b87c8179a0d21"
server_key_version: "kuduclusterkey@0"
I20260805 23:12:52.308530 6096 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260805 23:12:52.324060 6112 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:12:52.325632 6096 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260805 23:12:52.325795 6096 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/master-0/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/master-0/wal
uuid: "ba0742b645c94c13b3db716954ead8aa"
format_stamp: "Formatted at 2026-08-05 23:12:52 on dist-test-slave-ss4n"
server_key: "9b5e195756cfd7ca7f9f52451d7af6f6"
server_key_iv: "ab06c2ff8a4468c0670b87c8179a0d21"
server_key_version: "kuduclusterkey@0"
I20260805 23:12:52.325950 6096 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:12:52.342381 6096 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:12:52.346019 6096 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:12:52.346241 6096 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:12:52.354810 6096 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.254:44315
I20260805 23:12:52.354815 6166 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.254:44315 every 8 connection(s)
I20260805 23:12:52.355875 6096 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
I20260805 23:12:52.358888 6167 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:12:52.363127 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 6096
I20260805 23:12:52.363267 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/master-0/wal/instance
I20260805 23:12:52.363555 25971 external_mini_cluster.cc:1428] Setting key b174337d7ce5fde055b5786f3750dcdc
I20260805 23:12:52.365574 6167 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba0742b645c94c13b3db716954ead8aa: Bootstrap starting.
I20260805 23:12:52.368582 6167 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ba0742b645c94c13b3db716954ead8aa: Neither blocks nor log segments found. Creating new log.
I20260805 23:12:52.369439 6167 log.cc:824] T 00000000000000000000000000000000 P ba0742b645c94c13b3db716954ead8aa: Log is configured to *not* fsync() on all Append() calls
I20260805 23:12:52.371982 6167 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba0742b645c94c13b3db716954ead8aa: No bootstrap required, opened a new log
Aug 05 23:12:52 dist-test-slave-ss4n krb5kdc[5623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971572, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM
I20260805 23:12:52.376746 6167 raft_consensus.cc:348] T 00000000000000000000000000000000 P ba0742b645c94c13b3db716954ead8aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba0742b645c94c13b3db716954ead8aa" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 44315 } }
I20260805 23:12:52.377089 6167 raft_consensus.cc:374] T 00000000000000000000000000000000 P ba0742b645c94c13b3db716954ead8aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:12:52.377184 6167 raft_consensus.cc:729] T 00000000000000000000000000000000 P ba0742b645c94c13b3db716954ead8aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba0742b645c94c13b3db716954ead8aa, State: Initialized, Role: FOLLOWER
I20260805 23:12:52.377792 6167 consensus_queue.cc:260] T 00000000000000000000000000000000 P ba0742b645c94c13b3db716954ead8aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba0742b645c94c13b3db716954ead8aa" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 44315 } }
I20260805 23:12:52.378031 6167 raft_consensus.cc:388] T 00000000000000000000000000000000 P ba0742b645c94c13b3db716954ead8aa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:12:52.378119 6167 raft_consensus.cc:482] T 00000000000000000000000000000000 P ba0742b645c94c13b3db716954ead8aa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:12:52.378228 6167 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ba0742b645c94c13b3db716954ead8aa [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:12:52.379463 6167 raft_consensus.cc:504] T 00000000000000000000000000000000 P ba0742b645c94c13b3db716954ead8aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba0742b645c94c13b3db716954ead8aa" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 44315 } }
I20260805 23:12:52.379858 6167 leader_election.cc:302] T 00000000000000000000000000000000 P ba0742b645c94c13b3db716954ead8aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ba0742b645c94c13b3db716954ead8aa; no voters:
I20260805 23:12:52.380249 6167 leader_election.cc:288] T 00000000000000000000000000000000 P ba0742b645c94c13b3db716954ead8aa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 23:12:52.380820 6174 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ba0742b645c94c13b3db716954ead8aa [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:12:52.381100 6174 raft_consensus.cc:686] T 00000000000000000000000000000000 P ba0742b645c94c13b3db716954ead8aa [term 1 LEADER]: Becoming Leader. State: Replica: ba0742b645c94c13b3db716954ead8aa, State: Running, Role: LEADER
I20260805 23:12:52.381475 6174 consensus_queue.cc:237] T 00000000000000000000000000000000 P ba0742b645c94c13b3db716954ead8aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba0742b645c94c13b3db716954ead8aa" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 44315 } }
I20260805 23:12:52.381758 6167 sys_catalog.cc:564] T 00000000000000000000000000000000 P ba0742b645c94c13b3db716954ead8aa [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:12:52.382230 6171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:12:52.365266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58116 (local address 127.25.92.254:44315)
0805 23:12:52.365846 (+ 580us) server_negotiation.cc:206] Beginning negotiation
0805 23:12:52.365866 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0805 23:12:52.365913 (+ 47us) server_negotiation.cc:407] Connection header received
0805 23:12:52.366794 (+ 881us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:12:52.366818 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:12:52.367314 (+ 496us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:12:52.367715 (+ 401us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:12:52.368784 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:12:52.370255 (+ 1471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:12:52.371601 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:12:52.372323 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:12:52.372499 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:12:52.375842 (+ 3343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:12:52.375850 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:12:52.375873 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:12:52.375911 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:12:52.378244 (+ 2333us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:12:52.378875 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:12:52.378881 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:12:52.378887 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:12:52.379025 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:12:52.379363 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:12:52.379366 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:12:52.379368 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:12:52.379933 (+ 565us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:12:52.380177 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0805 23:12:52.380861 (+ 684us) server_negotiation.cc:299] Negotiation successful
0805 23:12:52.381175 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":391,"thread_start_us":194,"threads_started":1}
I20260805 23:12:52.383061 6174 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba0742b645c94c13b3db716954ead8aa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ba0742b645c94c13b3db716954ead8aa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba0742b645c94c13b3db716954ead8aa" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 44315 } } }
I20260805 23:12:52.383285 6174 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba0742b645c94c13b3db716954ead8aa [sys.catalog]: This master's current role is: LEADER
I20260805 23:12:52.383630 6174 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba0742b645c94c13b3db716954ead8aa [sys.catalog]: SysCatalogTable state changed. Reason: New leader ba0742b645c94c13b3db716954ead8aa. Latest consensus state: current_term: 1 leader_uuid: "ba0742b645c94c13b3db716954ead8aa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba0742b645c94c13b3db716954ead8aa" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 44315 } } }
I20260805 23:12:52.383771 6174 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba0742b645c94c13b3db716954ead8aa [sys.catalog]: This master's current role is: LEADER
I20260805 23:12:52.385296 6167 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260805 23:12:52.385979 6183 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:12:52.393174 6183 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:12:52.402382 6183 catalog_manager.cc:1347] Generated new cluster ID: 27805017ac4d4b06ac6977a0cd4f412b
I20260805 23:12:52.402463 6183 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:12:52.413095 6183 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:12:52.414629 6183 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:12:52.429154 6183 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ba0742b645c94c13b3db716954ead8aa: Generated new TSK 0
I20260805 23:12:52.430614 6183 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.92.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.92.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.92.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.193.keytab.
Entry for principal kudu/127.25.92.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.193.keytab.
I20260805 23:12:53.181459 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.25.92.212:46029/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.92.193:0
--local_ip_for_outbound_sockets=127.25.92.193
--webserver_interface=127.25.92.193
--webserver_port=0
--tserver_master_addrs=127.25.92.254:44315
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.193.keytab
--principal=kudu/127.25.92.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:35971
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:12:53.361678 6216 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:12:53.362229 6216 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:12:53.362377 6216 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:12:53.369642 6216 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:12:53.369844 6216 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:12:53.370033 6216 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.92.193
I20260805 23:12:53.376677 6216 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:35971
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.25.92.212:46029/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.193.keytab
--principal=kudu/127.25.92.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.92.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.92.193
--webserver_port=0
--tserver_master_addrs=127.25.92.254:44315
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6216
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.92.193
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:12:53.378077 6216 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:12:53.379454 6216 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:12:53.392310 6227 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:12:53.392460 6224 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:12:53.392575 6225 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:12:53.393923 6216 server_base.cc:1034] running on GCE node
I20260805 23:12:53.394516 6216 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:12:53.395422 6216 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:12:53.396637 6216 hybrid_clock.cc:648] HybridClock initialized: now 1785971573396610 us; error 45 us; skew 500 ppm
Aug 05 23:12:53 dist-test-slave-ss4n krb5kdc[5623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971573, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:12:53.400569 6216 init.cc:377] Logged in from keytab as kudu/127.25.92.193@KRBTEST.COM (short username kudu)
I20260805 23:12:53.401818 6216 webserver.cc:460] Webserver started at http://127.25.92.193:44507/ using document root <none> and password file <none>
I20260805 23:12:53.402601 6216 fs_manager.cc:359] Metadata directory not provided
I20260805 23:12:53.402699 6216 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:12:53.402940 6216 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 05 23:12:53 dist-test-slave-ss4n krb5kdc[5623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971573, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.193@KRBTEST.COM for HTTP/127.25.92.212@KRBTEST.COM
I20260805 23:12:53.419479 6216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ts-0/data/instance:
uuid: "9a8039f9d4884aee9bc77b7e45daa7ab"
format_stamp: "Formatted at 2026-08-05 23:12:53 on dist-test-slave-ss4n"
server_key: "2e5db5f4d89402ac7f34f6698f5a2dbf"
server_key_iv: "bf88a949ac238a202bb033496bcd8434"
server_key_version: "kuduclusterkey@0"
I20260805 23:12:53.420401 6216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ts-0/wal/instance:
uuid: "9a8039f9d4884aee9bc77b7e45daa7ab"
format_stamp: "Formatted at 2026-08-05 23:12:53 on dist-test-slave-ss4n"
server_key: "2e5db5f4d89402ac7f34f6698f5a2dbf"
server_key_iv: "bf88a949ac238a202bb033496bcd8434"
server_key_version: "kuduclusterkey@0"
I20260805 23:12:53.425259 6216 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260805 23:12:53.454093 6234 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:12:53.456118 6216 fs_manager.cc:725] Time spent opening block manager: real 0.015s user 0.003s sys 0.000s
I20260805 23:12:53.456360 6216 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ts-0/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ts-0/wal
uuid: "9a8039f9d4884aee9bc77b7e45daa7ab"
format_stamp: "Formatted at 2026-08-05 23:12:53 on dist-test-slave-ss4n"
server_key: "2e5db5f4d89402ac7f34f6698f5a2dbf"
server_key_iv: "bf88a949ac238a202bb033496bcd8434"
server_key_version: "kuduclusterkey@0"
I20260805 23:12:53.456607 6216 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:12:53.476547 6216 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:12:53.480768 6216 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:12:53.481117 6216 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:12:53.482079 6216 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:12:53.483513 6216 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:12:53.483623 6216 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:12:53.483716 6216 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:12:53.483771 6216 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:12:53.494907 6216 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.193:43873
I20260805 23:12:53.496203 6216 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ts-0/data/info.pb
I20260805 23:12:53.500499 6355 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.193:43873 every 8 connection(s)
I20260805 23:12:53.502750 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 6216
I20260805 23:12:53.502874 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ts-0/wal/instance
I20260805 23:12:53.503185 25971 external_mini_cluster.cc:1428] Setting key 04779fdef2be2886551edc43a5700795
Aug 05 23:12:53 dist-test-slave-ss4n krb5kdc[5623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971573, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.193@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.25.92.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.92.194@KRBTEST.COM" created.
I20260805 23:12:53.522538 6358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:12:53.503158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:44315 (local address 127.25.92.193:46715)
0805 23:12:53.505344 (+ 2186us) negotiation.cc:107] Waiting for socket to connect
0805 23:12:53.505408 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0805 23:12:53.506542 (+ 1134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:12:53.507495 (+ 953us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:12:53.507504 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:12:53.507861 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:12:53.508562 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:12:53.508581 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:12:53.509908 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:12:53.509912 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:12:53.511354 (+ 1442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:12:53.511367 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:12:53.512108 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:12:53.512112 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:12:53.512316 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:12:53.513062 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:12:53.513085 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:12:53.515013 (+ 1928us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:12:53.517718 (+ 2705us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:12:53.517727 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:12:53.517749 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:12:53.518200 (+ 451us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:12:53.518575 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:12:53.518579 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:12:53.518581 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:12:53.518783 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:12:53.519462 (+ 679us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:12:53.519469 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:12:53.520697 (+ 1228us) client_negotiation.cc:769] Sending connection context
0805 23:12:53.520933 (+ 236us) client_negotiation.cc:240] Negotiation successful
0805 23:12:53.521333 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1942,"thread_start_us":148,"threads_started":1}
I20260805 23:12:53.523233 6359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:12:53.504871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.193:46715 (local address 127.25.92.254:44315)
0805 23:12:53.507047 (+ 2176us) server_negotiation.cc:206] Beginning negotiation
0805 23:12:53.507052 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:12:53.507067 (+ 15us) server_negotiation.cc:407] Connection header received
0805 23:12:53.507110 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:12:53.507113 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:12:53.507160 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:12:53.508721 (+ 1561us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:12:53.508845 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:12:53.509750 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:12:53.511516 (+ 1766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:12:53.511871 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:12:53.515230 (+ 3359us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:12:53.515436 (+ 206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:12:53.515440 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:12:53.515442 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:12:53.515462 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:12:53.517528 (+ 2066us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:12:53.518345 (+ 817us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:12:53.518350 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:12:53.518352 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:12:53.518452 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:12:53.518902 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:12:53.518906 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:12:53.518908 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:12:53.519252 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:12:53.519377 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0805 23:12:53.522873 (+ 3496us) server_negotiation.cc:299] Negotiation successful
0805 23:12:53.523057 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2048,"thread_start_us":82,"threads_started":1}
I20260805 23:12:53.524969 6356 heartbeater.cc:347] Connected to a master server at 127.25.92.254:44315
I20260805 23:12:53.525252 6356 heartbeater.cc:464] Registering TS with master...
I20260805 23:12:53.525928 6356 heartbeater.cc:511] Master 127.25.92.254:44315 requested a full tablet report, sending...
I20260805 23:12:53.527902 6130 ts_manager.cc:194] Registered new tserver with Master: 9a8039f9d4884aee9bc77b7e45daa7ab (127.25.92.193:43873)
I20260805 23:12:53.529549 6130 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.92.193@KRBTEST.COM'} at 127.25.92.193:46715
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.92.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.194.keytab.
Entry for principal kudu/127.25.92.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.194.keytab.
I20260805 23:12:53.546065 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.25.92.212:46029/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.92.194:0
--local_ip_for_outbound_sockets=127.25.92.194
--webserver_interface=127.25.92.194
--webserver_port=0
--tserver_master_addrs=127.25.92.254:44315
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.194.keytab
--principal=kudu/127.25.92.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:35971
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 05 23:12:53 dist-test-slave-ss4n krb5kdc[5623](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1785971573, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260805 23:12:53.685129 6362 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:12:53.685433 6362 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:12:53.685532 6362 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:12:53.690042 6362 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:12:53.690136 6362 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:12:53.690263 6362 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.92.194
I20260805 23:12:53.696328 6362 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:35971
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.25.92.212:46029/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.194.keytab
--principal=kudu/127.25.92.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.92.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.92.194
--webserver_port=0
--tserver_master_addrs=127.25.92.254:44315
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6362
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.92.194
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:12:53.697652 6362 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:12:53.698709 6362 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:12:53.709326 6371 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:12:53.709829 6369 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:12:53.710104 6368 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:12:53.711544 6362 server_base.cc:1034] running on GCE node
I20260805 23:12:53.712132 6362 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:12:53.713064 6362 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:12:53.716379 6362 hybrid_clock.cc:648] HybridClock initialized: now 1785971573716307 us; error 121 us; skew 500 ppm
Aug 05 23:12:53 dist-test-slave-ss4n krb5kdc[5623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971573, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:12:53.720592 6362 init.cc:377] Logged in from keytab as kudu/127.25.92.194@KRBTEST.COM (short username kudu)
I20260805 23:12:53.721834 6362 webserver.cc:460] Webserver started at http://127.25.92.194:33169/ using document root <none> and password file <none>
I20260805 23:12:53.722486 6362 fs_manager.cc:359] Metadata directory not provided
I20260805 23:12:53.722586 6362 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:12:53.722846 6362 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 05 23:12:53 dist-test-slave-ss4n krb5kdc[5623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971573, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.194@KRBTEST.COM for HTTP/127.25.92.212@KRBTEST.COM
I20260805 23:12:53.740259 6362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ts-1/data/instance:
uuid: "162ee0db3de9468f864b397f56f554cd"
format_stamp: "Formatted at 2026-08-05 23:12:53 on dist-test-slave-ss4n"
server_key: "dda11f4b7e9b673540bc208b2865cdfc"
server_key_iv: "e1baa0996262c98028db45995559eb69"
server_key_version: "kuduclusterkey@0"
I20260805 23:12:53.741571 6362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ts-1/wal/instance:
uuid: "162ee0db3de9468f864b397f56f554cd"
format_stamp: "Formatted at 2026-08-05 23:12:53 on dist-test-slave-ss4n"
server_key: "dda11f4b7e9b673540bc208b2865cdfc"
server_key_iv: "e1baa0996262c98028db45995559eb69"
server_key_version: "kuduclusterkey@0"
I20260805 23:12:53.746913 6362 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260805 23:12:53.775326 6379 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:12:53.777127 6362 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260805 23:12:53.777284 6362 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ts-1/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ts-1/wal
uuid: "162ee0db3de9468f864b397f56f554cd"
format_stamp: "Formatted at 2026-08-05 23:12:53 on dist-test-slave-ss4n"
server_key: "dda11f4b7e9b673540bc208b2865cdfc"
server_key_iv: "e1baa0996262c98028db45995559eb69"
server_key_version: "kuduclusterkey@0"
I20260805 23:12:53.777428 6362 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:12:53.811199 6362 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:12:53.815797 6362 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:12:53.816084 6362 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:12:53.816883 6362 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:12:53.818089 6362 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:12:53.818172 6362 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:12:53.818269 6362 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:12:53.818317 6362 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:12:53.834060 6362 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.194:39189
I20260805 23:12:53.835449 6362 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ts-1/data/info.pb
I20260805 23:12:53.835916 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 6362
I20260805 23:12:53.836055 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ts-1/wal/instance
I20260805 23:12:53.836365 25971 external_mini_cluster.cc:1428] Setting key f78b356154b14d1f6a960aa1024fe7d6
I20260805 23:12:53.842157 6498 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.194:39189 every 8 connection(s)
WARNING: no policy specified for kudu/127.25.92.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.92.195@KRBTEST.COM" created.
Aug 05 23:12:53 dist-test-slave-ss4n krb5kdc[5623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971573, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.194@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM
I20260805 23:12:53.864807 6359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:12:53.847853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.194:41313 (local address 127.25.92.254:44315)
0805 23:12:53.848034 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0805 23:12:53.848041 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0805 23:12:53.849385 (+ 1344us) server_negotiation.cc:407] Connection header received
0805 23:12:53.851186 (+ 1801us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:12:53.851194 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:12:53.851272 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:12:53.851341 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:12:53.853610 (+ 2269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:12:53.854477 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:12:53.856159 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:12:53.856539 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:12:53.856684 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:12:53.860160 (+ 3476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:12:53.860165 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:12:53.860168 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:12:53.860203 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:12:53.862056 (+ 1853us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:12:53.862818 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:12:53.862822 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:12:53.862824 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:12:53.862919 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:12:53.863372 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:12:53.863376 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:12:53.863378 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:12:53.863632 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:12:53.863719 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0805 23:12:53.864464 (+ 745us) server_negotiation.cc:299] Negotiation successful
0805 23:12:53.864630 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260805 23:12:53.866590 6502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:12:53.848404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:44315 (local address 127.25.92.194:41313)
0805 23:12:53.849088 (+ 684us) negotiation.cc:107] Waiting for socket to connect
0805 23:12:53.849236 (+ 148us) client_negotiation.cc:174] Beginning negotiation
0805 23:12:53.850832 (+ 1596us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:12:53.851916 (+ 1084us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:12:53.851930 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:12:53.852349 (+ 419us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:12:53.853215 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:12:53.853240 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:12:53.854720 (+ 1480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:12:53.854724 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:12:53.855986 (+ 1262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:12:53.855999 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:12:53.856897 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:12:53.856901 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:12:53.857134 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:12:53.857971 (+ 837us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:12:53.857995 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:12:53.859870 (+ 1875us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:12:53.862241 (+ 2371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:12:53.862249 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:12:53.862270 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:12:53.862670 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:12:53.863034 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:12:53.863037 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:12:53.863039 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:12:53.863244 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:12:53.863794 (+ 550us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:12:53.863801 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:12:53.864161 (+ 360us) client_negotiation.cc:769] Sending connection context
0805 23:12:53.865010 (+ 849us) client_negotiation.cc:240] Negotiation successful
0805 23:12:53.865425 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":171,"threads_started":1}
I20260805 23:12:53.868096 6500 heartbeater.cc:347] Connected to a master server at 127.25.92.254:44315
I20260805 23:12:53.868403 6500 heartbeater.cc:464] Registering TS with master...
I20260805 23:12:53.869094 6500 heartbeater.cc:511] Master 127.25.92.254:44315 requested a full tablet report, sending...
I20260805 23:12:53.870427 6130 ts_manager.cc:194] Registered new tserver with Master: 162ee0db3de9468f864b397f56f554cd (127.25.92.194:39189)
I20260805 23:12:53.871246 6130 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.92.194@KRBTEST.COM'} at 127.25.92.194:41313
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.92.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.195.keytab.
Entry for principal kudu/127.25.92.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.195.keytab.
I20260805 23:12:53.889629 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.25.92.212:46029/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.92.195:0
--local_ip_for_outbound_sockets=127.25.92.195
--webserver_interface=127.25.92.195
--webserver_port=0
--tserver_master_addrs=127.25.92.254:44315
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.195.keytab
--principal=kudu/127.25.92.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:35971
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:12:54.085543 6504 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:12:54.086059 6504 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:12:54.086213 6504 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:12:54.092931 6504 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:12:54.093168 6504 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:12:54.093313 6504 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.92.195
I20260805 23:12:54.102478 6504 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:35971
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.25.92.212:46029/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.195.keytab
--principal=kudu/127.25.92.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.92.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.92.195
--webserver_port=0
--tserver_master_addrs=127.25.92.254:44315
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6504
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.92.195
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:12:54.104327 6504 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:12:54.106168 6504 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:12:54.120855 6514 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:12:54.120869 6511 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:12:54.121876 6504 server_base.cc:1034] running on GCE node
W20260805 23:12:54.126163 6510 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:12:54.126995 6504 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:12:54.127964 6504 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:12:54.143448 6504 hybrid_clock.cc:648] HybridClock initialized: now 1785971574143307 us; error 186 us; skew 500 ppm
Aug 05 23:12:54 dist-test-slave-ss4n krb5kdc[5623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971573, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.254@KRBTEST.COM for HTTP/127.25.92.212@KRBTEST.COM
Aug 05 23:12:54 dist-test-slave-ss4n krb5kdc[5623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971574, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:12:54.147621 6504 init.cc:377] Logged in from keytab as kudu/127.25.92.195@KRBTEST.COM (short username kudu)
I20260805 23:12:54.148926 6504 webserver.cc:460] Webserver started at http://127.25.92.195:46731/ using document root <none> and password file <none>
I20260805 23:12:54.149794 6504 fs_manager.cc:359] Metadata directory not provided
I20260805 23:12:54.149948 6504 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:12:54.150427 6504 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 05 23:12:54 dist-test-slave-ss4n krb5kdc[5623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971574, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.195@KRBTEST.COM for HTTP/127.25.92.212@KRBTEST.COM
I20260805 23:12:54.168092 6504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ts-2/data/instance:
uuid: "e4c6ea201d564bdc88361eb96db36bd7"
format_stamp: "Formatted at 2026-08-05 23:12:54 on dist-test-slave-ss4n"
server_key: "03bfb2f0e663d3d11171dc20c8b43683"
server_key_iv: "6467e40788d79acc84739115dc59c8e5"
server_key_version: "kuduclusterkey@0"
I20260805 23:12:54.175490 6504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ts-2/wal/instance:
uuid: "e4c6ea201d564bdc88361eb96db36bd7"
format_stamp: "Formatted at 2026-08-05 23:12:54 on dist-test-slave-ss4n"
server_key: "03bfb2f0e663d3d11171dc20c8b43683"
server_key_iv: "6467e40788d79acc84739115dc59c8e5"
server_key_version: "kuduclusterkey@0"
I20260805 23:12:54.194437 6504 fs_manager.cc:691] Time spent creating directory manager: real 0.018s user 0.005s sys 0.000s
I20260805 23:12:54.217677 6521 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:12:54.219704 6504 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260805 23:12:54.219941 6504 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ts-2/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ts-2/wal
uuid: "e4c6ea201d564bdc88361eb96db36bd7"
format_stamp: "Formatted at 2026-08-05 23:12:54 on dist-test-slave-ss4n"
server_key: "03bfb2f0e663d3d11171dc20c8b43683"
server_key_iv: "6467e40788d79acc84739115dc59c8e5"
server_key_version: "kuduclusterkey@0"
I20260805 23:12:54.220198 6504 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:12:54.248728 6504 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:12:54.252921 6504 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:12:54.253244 6504 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:12:54.254052 6504 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:12:54.255306 6504 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:12:54.255430 6504 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:12:54.255540 6504 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:12:54.255694 6504 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:12:54.267184 6504 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.195:46535
I20260805 23:12:54.268539 6504 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ts-2/data/info.pb
I20260805 23:12:54.274515 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 6504
I20260805 23:12:54.274657 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ts-2/wal/instance
I20260805 23:12:54.274989 25971 external_mini_cluster.cc:1428] Setting key 299598dacc49f9fb3b5bf60ae29e1ca9
I20260805 23:12:54.284520 6640 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.195:46535 every 8 connection(s)
Aug 05 23:12:54 dist-test-slave-ss4n krb5kdc[5623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971574, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.195@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM
I20260805 23:12:54.327397 6359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:12:54.288706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.195:50559 (local address 127.25.92.254:44315)
0805 23:12:54.288835 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0805 23:12:54.288840 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:12:54.293881 (+ 5041us) server_negotiation.cc:407] Connection header received
0805 23:12:54.295386 (+ 1505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:12:54.295391 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:12:54.295456 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:12:54.295525 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:12:54.313649 (+ 18124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:12:54.314539 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:12:54.316685 (+ 2146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:12:54.317047 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:12:54.317135 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:12:54.322588 (+ 5453us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:12:54.322593 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:12:54.322596 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:12:54.322630 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:12:54.324654 (+ 2024us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:12:54.325392 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:12:54.325396 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:12:54.325400 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:12:54.325522 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:12:54.326010 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:12:54.326014 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:12:54.326016 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:12:54.326256 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:12:54.326340 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0805 23:12:54.327085 (+ 745us) server_negotiation.cc:299] Negotiation successful
0805 23:12:54.327227 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260805 23:12:54.329108 6643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:12:54.287491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:44315 (local address 127.25.92.195:50559)
0805 23:12:54.293664 (+ 6173us) negotiation.cc:107] Waiting for socket to connect
0805 23:12:54.293755 (+ 91us) client_negotiation.cc:174] Beginning negotiation
0805 23:12:54.294992 (+ 1237us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:12:54.296022 (+ 1030us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:12:54.296033 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:12:54.296452 (+ 419us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:12:54.297197 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:12:54.297217 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:12:54.314779 (+ 17562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:12:54.314784 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:12:54.316486 (+ 1702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:12:54.316515 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:12:54.317390 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:12:54.317395 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:12:54.317671 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:12:54.319670 (+ 1999us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:12:54.319695 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:12:54.322287 (+ 2592us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:12:54.324826 (+ 2539us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:12:54.324833 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:12:54.324854 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:12:54.325274 (+ 420us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:12:54.325657 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:12:54.325660 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:12:54.325662 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:12:54.325881 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:12:54.326412 (+ 531us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:12:54.326419 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:12:54.326782 (+ 363us) client_negotiation.cc:769] Sending connection context
0805 23:12:54.327588 (+ 806us) client_negotiation.cc:240] Negotiation successful
0805 23:12:54.327971 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5808,"thread_start_us":189,"threads_started":1}
I20260805 23:12:54.330713 6641 heartbeater.cc:347] Connected to a master server at 127.25.92.254:44315
I20260805 23:12:54.331003 6641 heartbeater.cc:464] Registering TS with master...
I20260805 23:12:54.331676 6641 heartbeater.cc:511] Master 127.25.92.254:44315 requested a full tablet report, sending...
I20260805 23:12:54.332922 6130 ts_manager.cc:194] Registered new tserver with Master: e4c6ea201d564bdc88361eb96db36bd7 (127.25.92.195:46535)
I20260805 23:12:54.333740 6130 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.92.195@KRBTEST.COM'} at 127.25.92.195:50559
I20260805 23:12:54.337285 25971 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260805 23:12:54.338238 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 6216
I20260805 23:12:54.346565 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 6362
I20260805 23:12:54.356161 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 6504
I20260805 23:12:54.379045 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 6096
I20260805 23:12:54.389104 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.254:44315
--webserver_interface=127.25.92.254
--webserver_port=38815
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab
--principal=kudu/127.25.92.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:35971
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.92.254:44315
--ranger_config_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.25.92.212:46029/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.25.92.212:46029/kms with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:12:54.508437 6644 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:12:54.508755 6644 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:12:54.508797 6644 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:12:54.513093 6644 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:12:54.513175 6644 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:12:54.513195 6644 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:12:54.513238 6644 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:12:54.513284 6644 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:12:54.519403 6644 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:35971
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.25.92.212:46029/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.25.92.254:44315
--ranger_config_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab
--principal=kudu/127.25.92.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.254:44315
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.92.254
--webserver_port=38815
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6644
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:12:54.520606 6644 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:12:54.521773 6644 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:12:54.531090 6650 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:12:54.531100 6649 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:12:54.531142 6644 server_base.cc:1034] running on GCE node
W20260805 23:12:54.531107 6652 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:12:54.531971 6644 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:12:54.533124 6644 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:12:54.534292 6644 hybrid_clock.cc:648] HybridClock initialized: now 1785971574534266 us; error 38 us; skew 500 ppm
Aug 05 23:12:54 dist-test-slave-ss4n krb5kdc[5623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971574, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:12:54.537658 6644 init.cc:377] Logged in from keytab as kudu/127.25.92.254@KRBTEST.COM (short username kudu)
I20260805 23:12:54.538627 6644 webserver.cc:460] Webserver started at http://127.25.92.254:38815/ using document root <none> and password file <none>
I20260805 23:12:54.539211 6644 fs_manager.cc:359] Metadata directory not provided
I20260805 23:12:54.539306 6644 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 05 23:12:54 dist-test-slave-ss4n krb5kdc[5623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971574, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.254@KRBTEST.COM for HTTP/127.25.92.212@KRBTEST.COM
I20260805 23:12:54.596827 6644 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260805 23:12:54.599668 6660 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:12:54.601114 6644 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260805 23:12:54.601246 6644 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/master-0/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/master-0/wal
uuid: "ba0742b645c94c13b3db716954ead8aa"
format_stamp: "Formatted at 2026-08-05 23:12:52 on dist-test-slave-ss4n"
server_key: "9b5e195756cfd7ca7f9f52451d7af6f6"
server_key_iv: "ab06c2ff8a4468c0670b87c8179a0d21"
server_key_version: "kuduclusterkey@0"
I20260805 23:12:54.601964 6644 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:12:54.624691 6644 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:12:54.628109 6644 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:12:54.628285 6644 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:12:54.636973 6644 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.254:44315
I20260805 23:12:54.636981 6714 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.254:44315 every 8 connection(s)
I20260805 23:12:54.638303 6644 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
I20260805 23:12:54.638819 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 6644
I20260805 23:12:54.639382 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.25.92.212:46029/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.92.193:43873
--local_ip_for_outbound_sockets=127.25.92.193
--tserver_master_addrs=127.25.92.254:44315
--webserver_port=44507
--webserver_interface=127.25.92.193
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.193.keytab
--principal=kudu/127.25.92.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:35971
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.25.92.212:46029/kms with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260805 23:12:54.644754 6715 sys_catalog.cc:263] Verifying existing consensus state
I20260805 23:12:54.647886 6715 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba0742b645c94c13b3db716954ead8aa: Bootstrap starting.
I20260805 23:12:54.672899 6715 log.cc:824] T 00000000000000000000000000000000 P ba0742b645c94c13b3db716954ead8aa: Log is configured to *not* fsync() on all Append() calls
I20260805 23:12:54.682271 6715 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba0742b645c94c13b3db716954ead8aa: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260805 23:12:54.682794 6715 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba0742b645c94c13b3db716954ead8aa: Bootstrap complete.
I20260805 23:12:54.689047 6715 raft_consensus.cc:348] T 00000000000000000000000000000000 P ba0742b645c94c13b3db716954ead8aa [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba0742b645c94c13b3db716954ead8aa" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 44315 } }
I20260805 23:12:54.689754 6715 raft_consensus.cc:729] T 00000000000000000000000000000000 P ba0742b645c94c13b3db716954ead8aa [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba0742b645c94c13b3db716954ead8aa, State: Initialized, Role: FOLLOWER
I20260805 23:12:54.690371 6715 consensus_queue.cc:260] T 00000000000000000000000000000000 P ba0742b645c94c13b3db716954ead8aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba0742b645c94c13b3db716954ead8aa" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 44315 } }
I20260805 23:12:54.690538 6715 raft_consensus.cc:388] T 00000000000000000000000000000000 P ba0742b645c94c13b3db716954ead8aa [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:12:54.690623 6715 raft_consensus.cc:482] T 00000000000000000000000000000000 P ba0742b645c94c13b3db716954ead8aa [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:12:54.690729 6715 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ba0742b645c94c13b3db716954ead8aa [term 1 FOLLOWER]: Advancing to term 2
I20260805 23:12:54.693674 6715 raft_consensus.cc:504] T 00000000000000000000000000000000 P ba0742b645c94c13b3db716954ead8aa [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba0742b645c94c13b3db716954ead8aa" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 44315 } }
I20260805 23:12:54.694106 6715 leader_election.cc:302] T 00000000000000000000000000000000 P ba0742b645c94c13b3db716954ead8aa [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ba0742b645c94c13b3db716954ead8aa; no voters:
I20260805 23:12:54.694486 6715 leader_election.cc:288] T 00000000000000000000000000000000 P ba0742b645c94c13b3db716954ead8aa [CANDIDATE]: Term 2 election: Requested vote from peers
I20260805 23:12:54.694684 6719 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ba0742b645c94c13b3db716954ead8aa [term 2 FOLLOWER]: Leader election won for term 2
I20260805 23:12:54.695065 6719 raft_consensus.cc:686] T 00000000000000000000000000000000 P ba0742b645c94c13b3db716954ead8aa [term 2 LEADER]: Becoming Leader. State: Replica: ba0742b645c94c13b3db716954ead8aa, State: Running, Role: LEADER
I20260805 23:12:54.695474 6719 consensus_queue.cc:237] T 00000000000000000000000000000000 P ba0742b645c94c13b3db716954ead8aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba0742b645c94c13b3db716954ead8aa" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 44315 } }
I20260805 23:12:54.695962 6715 sys_catalog.cc:564] T 00000000000000000000000000000000 P ba0742b645c94c13b3db716954ead8aa [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:12:54.697228 6721 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba0742b645c94c13b3db716954ead8aa [sys.catalog]: SysCatalogTable state changed. Reason: New leader ba0742b645c94c13b3db716954ead8aa. Latest consensus state: current_term: 2 leader_uuid: "ba0742b645c94c13b3db716954ead8aa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba0742b645c94c13b3db716954ead8aa" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 44315 } } }
I20260805 23:12:54.697402 6721 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba0742b645c94c13b3db716954ead8aa [sys.catalog]: This master's current role is: LEADER
I20260805 23:12:54.697228 6720 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba0742b645c94c13b3db716954ead8aa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "ba0742b645c94c13b3db716954ead8aa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba0742b645c94c13b3db716954ead8aa" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 44315 } } }
I20260805 23:12:54.697634 6720 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba0742b645c94c13b3db716954ead8aa [sys.catalog]: This master's current role is: LEADER
I20260805 23:12:54.698608 6715 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260805 23:12:54.698920 6728 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:12:54.704663 6728 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:12:54.706283 6728 catalog_manager.cc:1259] Loaded cluster ID: 27805017ac4d4b06ac6977a0cd4f412b
I20260805 23:12:54.706557 6728 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:12:54.709308 6728 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:12:54.710386 6728 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ba0742b645c94c13b3db716954ead8aa: Loaded TSK: 0
I20260805 23:12:54.711812 6728 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260805 23:12:54.812132 6717 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:12:54.812525 6717 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:12:54.812592 6717 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:12:54.818801 6717 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:12:54.818892 6717 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:12:54.818984 6717 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.92.193
I20260805 23:12:54.827034 6717 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:35971
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.25.92.212:46029/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.193.keytab
--principal=kudu/127.25.92.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.92.193:43873
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.92.193
--webserver_port=44507
--tserver_master_addrs=127.25.92.254:44315
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6717
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.92.193
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:12:54.828477 6717 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:12:54.829798 6717 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260805 23:12:54.843456 6717 server_base.cc:1034] running on GCE node
W20260805 23:12:54.843372 6755 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:12:54.843703 6753 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:12:54.843722 6752 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:12:54.844405 6717 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:12:54.845223 6717 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:12:54.848546 6717 hybrid_clock.cc:648] HybridClock initialized: now 1785971574848449 us; error 105 us; skew 500 ppm
Aug 05 23:12:54 dist-test-slave-ss4n krb5kdc[5623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971574, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:12:54.852794 6717 init.cc:377] Logged in from keytab as kudu/127.25.92.193@KRBTEST.COM (short username kudu)
I20260805 23:12:54.854197 6717 webserver.cc:460] Webserver started at http://127.25.92.193:44507/ using document root <none> and password file <none>
I20260805 23:12:54.854966 6717 fs_manager.cc:359] Metadata directory not provided
I20260805 23:12:54.855093 6717 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 05 23:12:54 dist-test-slave-ss4n krb5kdc[5623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971574, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.193@KRBTEST.COM for HTTP/127.25.92.212@KRBTEST.COM
I20260805 23:12:54.885241 6717 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.003s sys 0.000s
I20260805 23:12:54.895821 6763 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:12:54.897708 6717 fs_manager.cc:725] Time spent opening block manager: real 0.011s user 0.003s sys 0.000s
I20260805 23:12:54.897861 6717 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ts-0/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ts-0/wal
uuid: "9a8039f9d4884aee9bc77b7e45daa7ab"
format_stamp: "Formatted at 2026-08-05 23:12:53 on dist-test-slave-ss4n"
server_key: "2e5db5f4d89402ac7f34f6698f5a2dbf"
server_key_iv: "bf88a949ac238a202bb033496bcd8434"
server_key_version: "kuduclusterkey@0"
I20260805 23:12:54.898609 6717 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:12:54.929200 6717 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:12:54.932703 6717 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:12:54.932928 6717 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:12:54.933643 6717 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:12:54.934772 6717 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:12:54.934851 6717 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:12:54.934922 6717 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:12:54.934948 6717 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:12:54.945273 6717 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.193:43873
I20260805 23:12:54.946043 6885 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.193:43873 every 8 connection(s)
I20260805 23:12:54.946661 6717 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ts-0/data/info.pb
I20260805 23:12:54.949811 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 6717
I20260805 23:12:54.950328 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.25.92.212:46029/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.92.194:39189
--local_ip_for_outbound_sockets=127.25.92.194
--tserver_master_addrs=127.25.92.254:44315
--webserver_port=33169
--webserver_interface=127.25.92.194
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.194.keytab
--principal=kudu/127.25.92.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:35971
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.25.92.212:46029/kms with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 05 23:12:54 dist-test-slave-ss4n krb5kdc[5623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971574, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.193@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM
I20260805 23:12:54.969374 6888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:12:54.951084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:44315 (local address 127.25.92.193:49459)
0805 23:12:54.954065 (+ 2981us) negotiation.cc:107] Waiting for socket to connect
0805 23:12:54.954115 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0805 23:12:54.954989 (+ 874us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:12:54.956422 (+ 1433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:12:54.956431 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:12:54.956813 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:12:54.957509 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:12:54.957535 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:12:54.959404 (+ 1869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:12:54.959407 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:12:54.960280 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:12:54.960289 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:12:54.961105 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:12:54.961109 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:12:54.961316 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:12:54.962073 (+ 757us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:12:54.962091 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:12:54.963520 (+ 1429us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:12:54.965938 (+ 2418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:12:54.965944 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:12:54.965959 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:12:54.966229 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:12:54.966640 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:12:54.966645 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:12:54.966647 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:12:54.966795 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:12:54.967457 (+ 662us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:12:54.967464 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:12:54.967780 (+ 316us) client_negotiation.cc:769] Sending connection context
0805 23:12:54.967976 (+ 196us) client_negotiation.cc:240] Negotiation successful
0805 23:12:54.968333 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2769,"thread_start_us":159,"threads_started":1}
I20260805 23:12:54.969362 6890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:12:54.952616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.193:49459 (local address 127.25.92.254:44315)
0805 23:12:54.953287 (+ 671us) server_negotiation.cc:206] Beginning negotiation
0805 23:12:54.953312 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0805 23:12:54.954262 (+ 950us) server_negotiation.cc:407] Connection header received
0805 23:12:54.955457 (+ 1195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:12:54.955472 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:12:54.955889 (+ 417us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:12:54.956285 (+ 396us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:12:54.957764 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:12:54.959241 (+ 1477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:12:54.960432 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:12:54.960911 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:12:54.961102 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:12:54.963746 (+ 2644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:12:54.963752 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:12:54.963766 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:12:54.963793 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:12:54.965781 (+ 1988us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:12:54.966366 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:12:54.966372 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:12:54.966377 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:12:54.966543 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:12:54.966900 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:12:54.966903 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:12:54.966905 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:12:54.967341 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:12:54.967489 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0805 23:12:54.968124 (+ 635us) server_negotiation.cc:299] Negotiation successful
0805 23:12:54.968375 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":502,"thread_start_us":121,"threads_started":1}
I20260805 23:12:54.972079 6886 heartbeater.cc:347] Connected to a master server at 127.25.92.254:44315
I20260805 23:12:54.972334 6886 heartbeater.cc:464] Registering TS with master...
I20260805 23:12:54.972894 6886 heartbeater.cc:511] Master 127.25.92.254:44315 requested a full tablet report, sending...
W20260805 23:12:54.974992 6886 heartbeater.cc:644] Failed to heartbeat to 127.25.92.254:44315 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260805 23:12:54.975128 6886 heartbeater.cc:464] Registering TS with master...
I20260805 23:12:54.975317 6886 heartbeater.cc:511] Master 127.25.92.254:44315 requested a full tablet report, sending...
I20260805 23:12:54.976154 6886 heartbeater.cc:464] Registering TS with master...
I20260805 23:12:54.976352 6886 heartbeater.cc:511] Master 127.25.92.254:44315 requested a full tablet report, sending...
W20260805 23:12:54.977114 6886 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260805 23:12:55.130241 6889 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:12:55.130643 6889 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:12:55.130718 6889 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:12:55.136586 6889 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:12:55.136708 6889 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:12:55.136821 6889 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.92.194
I20260805 23:12:55.145226 6889 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:35971
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.25.92.212:46029/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.194.keytab
--principal=kudu/127.25.92.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.92.194:39189
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.92.194
--webserver_port=33169
--tserver_master_addrs=127.25.92.254:44315
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6889
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.92.194
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:12:55.146821 6889 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:12:55.148298 6889 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:12:55.162128 6898 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:12:55.162559 6895 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:12:55.165037 6889 server_base.cc:1034] running on GCE node
W20260805 23:12:55.167651 6896 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:12:55.168489 6889 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:12:55.169548 6889 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:12:55.172904 6889 hybrid_clock.cc:648] HybridClock initialized: now 1785971575171766 us; error 1148 us; skew 500 ppm
Aug 05 23:12:55 dist-test-slave-ss4n krb5kdc[5623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971575, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:12:55.177454 6889 init.cc:377] Logged in from keytab as kudu/127.25.92.194@KRBTEST.COM (short username kudu)
I20260805 23:12:55.179024 6889 webserver.cc:460] Webserver started at http://127.25.92.194:33169/ using document root <none> and password file <none>
I20260805 23:12:55.179852 6889 fs_manager.cc:359] Metadata directory not provided
I20260805 23:12:55.179988 6889 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 05 23:12:55 dist-test-slave-ss4n krb5kdc[5623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971575, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.194@KRBTEST.COM for HTTP/127.25.92.212@KRBTEST.COM
I20260805 23:12:55.209685 6889 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260805 23:12:55.213541 6906 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:12:55.215392 6889 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260805 23:12:55.215566 6889 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ts-1/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ts-1/wal
uuid: "162ee0db3de9468f864b397f56f554cd"
format_stamp: "Formatted at 2026-08-05 23:12:53 on dist-test-slave-ss4n"
server_key: "dda11f4b7e9b673540bc208b2865cdfc"
server_key_iv: "e1baa0996262c98028db45995559eb69"
server_key_version: "kuduclusterkey@0"
I20260805 23:12:55.216468 6889 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:12:55.243634 6889 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:12:55.248785 6889 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:12:55.249060 6889 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:12:55.249900 6889 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:12:55.251133 6889 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:12:55.251222 6889 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:12:55.251287 6889 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:12:55.251377 6889 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260805 23:12:55.264391 6889 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.194:39189
I20260805 23:12:55.264411 7024 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.194:39189 every 8 connection(s)
I20260805 23:12:55.265960 6889 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ts-1/data/info.pb
I20260805 23:12:55.268956 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 6889
I20260805 23:12:55.269520 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.25.92.212:46029/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.92.195:46535
--local_ip_for_outbound_sockets=127.25.92.195
--tserver_master_addrs=127.25.92.254:44315
--webserver_port=46731
--webserver_interface=127.25.92.195
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.195.keytab
--principal=kudu/127.25.92.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:35971
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.25.92.212:46029/kms with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 05 23:12:55 dist-test-slave-ss4n krb5kdc[5623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971575, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.194@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM
I20260805 23:12:55.286819 6890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:12:55.268413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.194:32949 (local address 127.25.92.254:44315)
0805 23:12:55.268620 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0805 23:12:55.268626 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:12:55.270643 (+ 2017us) server_negotiation.cc:407] Connection header received
0805 23:12:55.272099 (+ 1456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:12:55.272104 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:12:55.272173 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:12:55.272288 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:12:55.274132 (+ 1844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:12:55.274997 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:12:55.277202 (+ 2205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:12:55.277614 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:12:55.277731 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:12:55.281446 (+ 3715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:12:55.281450 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:12:55.281453 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:12:55.281510 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:12:55.283625 (+ 2115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:12:55.284466 (+ 841us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:12:55.284470 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:12:55.284472 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:12:55.284582 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:12:55.285128 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:12:55.285135 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:12:55.285139 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:12:55.285457 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:12:55.285614 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0805 23:12:55.286430 (+ 816us) server_negotiation.cc:299] Negotiation successful
0805 23:12:55.286621 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260805 23:12:55.289165 7027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:12:55.268927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:44315 (local address 127.25.92.194:32949)
0805 23:12:55.270385 (+ 1458us) negotiation.cc:107] Waiting for socket to connect
0805 23:12:55.270459 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0805 23:12:55.271785 (+ 1326us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:12:55.272576 (+ 791us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:12:55.272588 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:12:55.273040 (+ 452us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:12:55.273901 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:12:55.273925 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:12:55.275188 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:12:55.275195 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:12:55.277021 (+ 1826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:12:55.277040 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:12:55.277820 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:12:55.277828 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:12:55.278115 (+ 287us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:12:55.278911 (+ 796us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:12:55.278944 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:12:55.281189 (+ 2245us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:12:55.283795 (+ 2606us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:12:55.283805 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:12:55.283831 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:12:55.284321 (+ 490us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:12:55.284711 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:12:55.284718 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:12:55.284722 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:12:55.284971 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:12:55.285712 (+ 741us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:12:55.285719 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:12:55.286104 (+ 385us) client_negotiation.cc:769] Sending connection context
0805 23:12:55.287584 (+ 1480us) client_negotiation.cc:240] Negotiation successful
0805 23:12:55.288007 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1175,"thread_start_us":165,"threads_started":1}
I20260805 23:12:55.290908 7025 heartbeater.cc:347] Connected to a master server at 127.25.92.254:44315
I20260805 23:12:55.291121 7025 heartbeater.cc:464] Registering TS with master...
I20260805 23:12:55.291752 7025 heartbeater.cc:511] Master 127.25.92.254:44315 requested a full tablet report, sending...
W20260805 23:12:55.293820 7025 heartbeater.cc:644] Failed to heartbeat to 127.25.92.254:44315 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260805 23:12:55.293987 7025 heartbeater.cc:464] Registering TS with master...
I20260805 23:12:55.294215 7025 heartbeater.cc:511] Master 127.25.92.254:44315 requested a full tablet report, sending...
I20260805 23:12:55.295241 7025 heartbeater.cc:464] Registering TS with master...
I20260805 23:12:55.295434 7025 heartbeater.cc:511] Master 127.25.92.254:44315 requested a full tablet report, sending...
W20260805 23:12:55.296303 7025 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260805 23:12:55.427270 7028 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:12:55.427616 7028 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:12:55.427747 7028 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:12:55.433079 7028 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:12:55.433250 7028 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:12:55.433387 7028 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.92.195
I20260805 23:12:55.440742 7028 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:35971
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.25.92.212:46029/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.195.keytab
--principal=kudu/127.25.92.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.92.195:46535
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.92.195
--webserver_port=46731
--tserver_master_addrs=127.25.92.254:44315
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7028
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.92.195
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:12:55.442015 7028 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:12:55.443127 7028 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:12:55.455572 7033 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:12:55.456475 7028 server_base.cc:1034] running on GCE node
W20260805 23:12:55.456802 7036 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:12:55.457059 7034 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:12:55.457650 7028 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:12:55.458434 7028 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:12:55.459662 7028 hybrid_clock.cc:648] HybridClock initialized: now 1785971575459647 us; error 32 us; skew 500 ppm
Aug 05 23:12:55 dist-test-slave-ss4n krb5kdc[5623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971575, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:12:55.463797 7028 init.cc:377] Logged in from keytab as kudu/127.25.92.195@KRBTEST.COM (short username kudu)
I20260805 23:12:55.465102 7028 webserver.cc:460] Webserver started at http://127.25.92.195:46731/ using document root <none> and password file <none>
I20260805 23:12:55.465899 7028 fs_manager.cc:359] Metadata directory not provided
I20260805 23:12:55.466037 7028 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 05 23:12:55 dist-test-slave-ss4n krb5kdc[5623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971575, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.195@KRBTEST.COM for HTTP/127.25.92.212@KRBTEST.COM
I20260805 23:12:55.487718 7028 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.003s sys 0.001s
I20260805 23:12:55.491314 7044 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:12:55.493149 7028 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260805 23:12:55.493297 7028 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ts-2/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ts-2/wal
uuid: "e4c6ea201d564bdc88361eb96db36bd7"
format_stamp: "Formatted at 2026-08-05 23:12:54 on dist-test-slave-ss4n"
server_key: "03bfb2f0e663d3d11171dc20c8b43683"
server_key_iv: "6467e40788d79acc84739115dc59c8e5"
server_key_version: "kuduclusterkey@0"
I20260805 23:12:55.494366 7028 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:12:55.521843 7028 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:12:55.526299 7028 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:12:55.526542 7028 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:12:55.527355 7028 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:12:55.528635 7028 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:12:55.528717 7028 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:12:55.528825 7028 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:12:55.528858 7028 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:12:55.541164 7028 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.195:46535
I20260805 23:12:55.541199 7166 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.195:46535 every 8 connection(s)
I20260805 23:12:55.542701 7028 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ts-2/data/info.pb
I20260805 23:12:55.551190 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 7028
Aug 05 23:12:55 dist-test-slave-ss4n krb5kdc[5623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971575, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.195@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM
I20260805 23:12:55.560367 6890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:12:55.545641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.195:32895 (local address 127.25.92.254:44315)
0805 23:12:55.545827 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0805 23:12:55.545833 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:12:55.546215 (+ 382us) server_negotiation.cc:407] Connection header received
0805 23:12:55.547321 (+ 1106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:12:55.547325 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:12:55.547390 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:12:55.547480 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:12:55.548943 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:12:55.549703 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:12:55.551040 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:12:55.551406 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:12:55.551547 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:12:55.555683 (+ 4136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:12:55.555687 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:12:55.555690 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:12:55.555720 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:12:55.557689 (+ 1969us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:12:55.558281 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:12:55.558283 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:12:55.558284 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:12:55.558354 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:12:55.558815 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:12:55.558819 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:12:55.558821 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:12:55.559075 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:12:55.559181 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0805 23:12:55.559990 (+ 809us) server_negotiation.cc:299] Negotiation successful
0805 23:12:55.560148 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260805 23:12:55.561394 7169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:12:55.545393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:44315 (local address 127.25.92.195:32895)
0805 23:12:55.545993 (+ 600us) negotiation.cc:107] Waiting for socket to connect
0805 23:12:55.546080 (+ 87us) client_negotiation.cc:174] Beginning negotiation
0805 23:12:55.547074 (+ 994us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:12:55.547767 (+ 693us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:12:55.547775 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:12:55.548069 (+ 294us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:12:55.548742 (+ 673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:12:55.548764 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:12:55.549870 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:12:55.549874 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:12:55.550858 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:12:55.550874 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:12:55.551603 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:12:55.551610 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:12:55.551859 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:12:55.552588 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:12:55.552611 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:12:55.555445 (+ 2834us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:12:55.557843 (+ 2398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:12:55.557849 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:12:55.557866 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:12:55.558169 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:12:55.558454 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:12:55.558460 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:12:55.558464 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:12:55.558684 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:12:55.559249 (+ 565us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:12:55.559258 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:12:55.559658 (+ 400us) client_negotiation.cc:769] Sending connection context
0805 23:12:55.559913 (+ 255us) client_negotiation.cc:240] Negotiation successful
0805 23:12:55.560283 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":157,"threads_started":1}
I20260805 23:12:55.563053 7167 heartbeater.cc:347] Connected to a master server at 127.25.92.254:44315
I20260805 23:12:55.563331 7167 heartbeater.cc:464] Registering TS with master...
I20260805 23:12:55.563973 7167 heartbeater.cc:511] Master 127.25.92.254:44315 requested a full tablet report, sending...
I20260805 23:12:55.564350 7171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:12:55.552060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58122 (local address 127.25.92.254:44315)
0805 23:12:55.552409 (+ 349us) server_negotiation.cc:206] Beginning negotiation
0805 23:12:55.552418 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0805 23:12:55.552761 (+ 343us) server_negotiation.cc:407] Connection header received
0805 23:12:55.552925 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:12:55.552928 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:12:55.552988 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:12:55.553084 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:12:55.554089 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:12:55.554965 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:12:55.556147 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:12:55.556507 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:12:55.556630 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:12:55.558577 (+ 1947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:12:55.558580 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:12:55.558583 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:12:55.558606 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:12:55.560289 (+ 1683us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:12:55.562758 (+ 2469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:12:55.562764 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:12:55.562766 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:12:55.562865 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:12:55.563264 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:12:55.563267 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:12:55.563269 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:12:55.563537 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:12:55.563767 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0805 23:12:55.563972 (+ 205us) server_negotiation.cc:299] Negotiation successful
0805 23:12:55.564124 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":256,"thread_start_us":153,"threads_started":1}
I20260805 23:12:55.565558 6678 ts_manager.cc:194] Registered new tserver with Master: e4c6ea201d564bdc88361eb96db36bd7 (127.25.92.195:46535)
I20260805 23:12:55.567173 6678 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.92.195@KRBTEST.COM'} at 127.25.92.195:32895
I20260805 23:12:55.977471 6886 heartbeater.cc:464] Registering TS with master...
I20260805 23:12:55.978371 6886 heartbeater.cc:511] Master 127.25.92.254:44315 requested a full tablet report, sending...
I20260805 23:12:55.981371 6678 ts_manager.cc:194] Registered new tserver with Master: 9a8039f9d4884aee9bc77b7e45daa7ab (127.25.92.193:43873)
I20260805 23:12:55.983815 6678 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.92.193@KRBTEST.COM'} at 127.25.92.193:49459
Aug 05 23:12:56 dist-test-slave-ss4n krb5kdc[5623](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1785971576, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:12:56.296763 7025 heartbeater.cc:464] Registering TS with master...
I20260805 23:12:56.297149 7025 heartbeater.cc:511] Master 127.25.92.254:44315 requested a full tablet report, sending...
I20260805 23:12:56.298336 6678 ts_manager.cc:194] Registered new tserver with Master: 162ee0db3de9468f864b397f56f554cd (127.25.92.194:39189)
I20260805 23:12:56.299229 6678 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.92.194@KRBTEST.COM'} at 127.25.92.194:32949
I20260805 23:12:56.299906 25971 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260805 23:12:56.315043 7181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:12:56.301808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58126 (local address 127.25.92.254:44315)
0805 23:12:56.302406 (+ 598us) server_negotiation.cc:206] Beginning negotiation
0805 23:12:56.302413 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0805 23:12:56.302669 (+ 256us) server_negotiation.cc:407] Connection header received
0805 23:12:56.302842 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:12:56.302846 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:12:56.302904 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:12:56.303079 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:12:56.303776 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:12:56.304537 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:12:56.305701 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:12:56.305960 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:12:56.306089 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:12:56.311064 (+ 4975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:12:56.311068 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:12:56.311071 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:12:56.311103 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:12:56.312719 (+ 1616us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:12:56.313197 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:12:56.313200 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:12:56.313202 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:12:56.313296 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:12:56.313643 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:12:56.313646 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:12:56.313648 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:12:56.313874 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:12:56.314005 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0805 23:12:56.314718 (+ 713us) server_negotiation.cc:299] Negotiation successful
0805 23:12:56.314855 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":507,"thread_start_us":118,"threads_started":1}
I20260805 23:12:56.319579 6678 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58126:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260805 23:12:56.323736 6678 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260805 23:12:56.345762 7192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:12:56.338495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54650 (local address 127.25.92.194:39189)
0805 23:12:56.338808 (+ 313us) server_negotiation.cc:206] Beginning negotiation
0805 23:12:56.338814 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:12:56.338840 (+ 26us) server_negotiation.cc:407] Connection header received
0805 23:12:56.340128 (+ 1288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:12:56.340135 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:12:56.340330 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:12:56.340730 (+ 400us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:12:56.340888 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:12:56.341985 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:12:56.343737 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:12:56.344609 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:12:56.345219 (+ 610us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:12:56.345374 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0805 23:12:56.345429 (+ 55us) server_negotiation.cc:299] Negotiation successful
0805 23:12:56.345566 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":221,"thread_start_us":78,"threads_started":1}
I20260805 23:12:56.346329 7188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:12:56.336629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.194:39189 (local address 127.0.0.1:54650)
0805 23:12:56.337722 (+ 1093us) negotiation.cc:107] Waiting for socket to connect
0805 23:12:56.337781 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0805 23:12:56.337913 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:12:56.340473 (+ 2560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:12:56.340477 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:12:56.340493 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:12:56.340585 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:12:56.340591 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:12:56.342125 (+ 1534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:12:56.342128 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:12:56.343585 (+ 1457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:12:56.343595 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:12:56.344755 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:12:56.344759 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:12:56.344866 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:12:56.344907 (+ 41us) client_negotiation.cc:769] Sending connection context
0805 23:12:56.345002 (+ 95us) client_negotiation.cc:240] Negotiation successful
0805 23:12:56.346192 (+ 1190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":897,"thread_start_us":128,"threads_started":1}
I20260805 23:12:56.346431 7190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:12:56.337929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44648 (local address 127.25.92.193:43873)
0805 23:12:56.338341 (+ 412us) server_negotiation.cc:206] Beginning negotiation
0805 23:12:56.338349 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0805 23:12:56.339466 (+ 1117us) server_negotiation.cc:407] Connection header received
0805 23:12:56.339553 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:12:56.339560 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:12:56.339766 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:12:56.339938 (+ 172us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:12:56.340245 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:12:56.341544 (+ 1299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:12:56.343870 (+ 2326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:12:56.344616 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:12:56.344728 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:12:56.344892 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0805 23:12:56.346176 (+ 1284us) server_negotiation.cc:299] Negotiation successful
0805 23:12:56.346281 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":103,"threads_started":1}
I20260805 23:12:56.346889 7189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:12:56.337033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.193:43873 (local address 127.0.0.1:44648)
0805 23:12:56.339035 (+ 2002us) negotiation.cc:107] Waiting for socket to connect
0805 23:12:56.339049 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0805 23:12:56.339155 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:12:56.339925 (+ 770us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:12:56.339931 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:12:56.339954 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:12:56.340109 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:12:56.340116 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:12:56.342603 (+ 2487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:12:56.342606 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:12:56.343749 (+ 1143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:12:56.343756 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:12:56.344950 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:12:56.344953 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:12:56.345990 (+ 1037us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:12:56.346013 (+ 23us) client_negotiation.cc:769] Sending connection context
0805 23:12:56.346083 (+ 70us) client_negotiation.cc:240] Negotiation successful
0805 23:12:56.346724 (+ 641us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1931,"mutex_wait_us":526,"thread_start_us":112,"threads_started":1}
I20260805 23:12:56.350418 6815 tablet_service.cc:1467] Processing CreateTablet for tablet 9988db2ba1db43509f01fb1ddc78fc25 (DEFAULT_TABLE table=test-table [id=adf01a54dcb04783a9b1625d2d12d05e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:12:56.351060 6956 tablet_service.cc:1467] Processing CreateTablet for tablet 9988db2ba1db43509f01fb1ddc78fc25 (DEFAULT_TABLE table=test-table [id=adf01a54dcb04783a9b1625d2d12d05e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:12:56.351701 6815 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9988db2ba1db43509f01fb1ddc78fc25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:12:56.352041 6956 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9988db2ba1db43509f01fb1ddc78fc25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:12:56.353577 7188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:12:56.337633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.195:46535 (local address 127.0.0.1:39716)
0805 23:12:56.347212 (+ 9579us) negotiation.cc:107] Waiting for socket to connect
0805 23:12:56.347226 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0805 23:12:56.347306 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:12:56.348025 (+ 719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:12:56.348027 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:12:56.348041 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:12:56.348109 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:12:56.348113 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:12:56.350472 (+ 2359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:12:56.350476 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:12:56.351606 (+ 1130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:12:56.351616 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:12:56.352915 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:12:56.352919 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:12:56.353009 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:12:56.353036 (+ 27us) client_negotiation.cc:769] Sending connection context
0805 23:12:56.353112 (+ 76us) client_negotiation.cc:240] Negotiation successful
0805 23:12:56.353196 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9511,"thread_start_us":91,"threads_started":1}
I20260805 23:12:56.355108 7191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:12:56.338166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39716 (local address 127.25.92.195:46535)
0805 23:12:56.339393 (+ 1227us) server_negotiation.cc:206] Beginning negotiation
0805 23:12:56.339400 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0805 23:12:56.347413 (+ 8013us) server_negotiation.cc:407] Connection header received
0805 23:12:56.347502 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:12:56.347507 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:12:56.347730 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:12:56.347882 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:12:56.348242 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:12:56.350327 (+ 2085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:12:56.351761 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:12:56.352592 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:12:56.352701 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:12:56.352846 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0805 23:12:56.354840 (+ 1994us) server_negotiation.cc:299] Negotiation successful
0805 23:12:56.354963 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1117,"thread_start_us":81,"threads_started":1}
I20260805 23:12:56.358165 7100 tablet_service.cc:1467] Processing CreateTablet for tablet 9988db2ba1db43509f01fb1ddc78fc25 (DEFAULT_TABLE table=test-table [id=adf01a54dcb04783a9b1625d2d12d05e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:12:56.359319 7100 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9988db2ba1db43509f01fb1ddc78fc25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:12:56.361759 7194 tablet_bootstrap.cc:492] T 9988db2ba1db43509f01fb1ddc78fc25 P 162ee0db3de9468f864b397f56f554cd: Bootstrap starting.
I20260805 23:12:56.364260 7194 tablet_bootstrap.cc:654] T 9988db2ba1db43509f01fb1ddc78fc25 P 162ee0db3de9468f864b397f56f554cd: Neither blocks nor log segments found. Creating new log.
I20260805 23:12:56.365187 7194 log.cc:824] T 9988db2ba1db43509f01fb1ddc78fc25 P 162ee0db3de9468f864b397f56f554cd: Log is configured to *not* fsync() on all Append() calls
I20260805 23:12:56.365823 7193 tablet_bootstrap.cc:492] T 9988db2ba1db43509f01fb1ddc78fc25 P 9a8039f9d4884aee9bc77b7e45daa7ab: Bootstrap starting.
I20260805 23:12:56.368214 7193 tablet_bootstrap.cc:654] T 9988db2ba1db43509f01fb1ddc78fc25 P 9a8039f9d4884aee9bc77b7e45daa7ab: Neither blocks nor log segments found. Creating new log.
I20260805 23:12:56.369144 7193 log.cc:824] T 9988db2ba1db43509f01fb1ddc78fc25 P 9a8039f9d4884aee9bc77b7e45daa7ab: Log is configured to *not* fsync() on all Append() calls
I20260805 23:12:56.372215 7193 tablet_bootstrap.cc:492] T 9988db2ba1db43509f01fb1ddc78fc25 P 9a8039f9d4884aee9bc77b7e45daa7ab: No bootstrap required, opened a new log
I20260805 23:12:56.372412 7193 ts_tablet_manager.cc:1397] T 9988db2ba1db43509f01fb1ddc78fc25 P 9a8039f9d4884aee9bc77b7e45daa7ab: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.002s
I20260805 23:12:56.374459 7194 tablet_bootstrap.cc:492] T 9988db2ba1db43509f01fb1ddc78fc25 P 162ee0db3de9468f864b397f56f554cd: No bootstrap required, opened a new log
I20260805 23:12:56.374711 7194 ts_tablet_manager.cc:1397] T 9988db2ba1db43509f01fb1ddc78fc25 P 162ee0db3de9468f864b397f56f554cd: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.003s
I20260805 23:12:56.376101 7193 raft_consensus.cc:348] T 9988db2ba1db43509f01fb1ddc78fc25 P 9a8039f9d4884aee9bc77b7e45daa7ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "162ee0db3de9468f864b397f56f554cd" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 39189 } } peers { permanent_uuid: "e4c6ea201d564bdc88361eb96db36bd7" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 46535 } } peers { permanent_uuid: "9a8039f9d4884aee9bc77b7e45daa7ab" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 43873 } }
I20260805 23:12:56.376400 7193 raft_consensus.cc:374] T 9988db2ba1db43509f01fb1ddc78fc25 P 9a8039f9d4884aee9bc77b7e45daa7ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:12:56.376451 7193 raft_consensus.cc:729] T 9988db2ba1db43509f01fb1ddc78fc25 P 9a8039f9d4884aee9bc77b7e45daa7ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a8039f9d4884aee9bc77b7e45daa7ab, State: Initialized, Role: FOLLOWER
I20260805 23:12:56.376994 7193 consensus_queue.cc:260] T 9988db2ba1db43509f01fb1ddc78fc25 P 9a8039f9d4884aee9bc77b7e45daa7ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "162ee0db3de9468f864b397f56f554cd" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 39189 } } peers { permanent_uuid: "e4c6ea201d564bdc88361eb96db36bd7" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 46535 } } peers { permanent_uuid: "9a8039f9d4884aee9bc77b7e45daa7ab" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 43873 } }
I20260805 23:12:56.378284 7193 ts_tablet_manager.cc:1428] T 9988db2ba1db43509f01fb1ddc78fc25 P 9a8039f9d4884aee9bc77b7e45daa7ab: Time spent starting tablet: real 0.006s user 0.002s sys 0.005s
I20260805 23:12:56.378677 6886 heartbeater.cc:503] Master 127.25.92.254:44315 was elected leader, sending a full tablet report...
I20260805 23:12:56.379330 7194 raft_consensus.cc:348] T 9988db2ba1db43509f01fb1ddc78fc25 P 162ee0db3de9468f864b397f56f554cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "162ee0db3de9468f864b397f56f554cd" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 39189 } } peers { permanent_uuid: "e4c6ea201d564bdc88361eb96db36bd7" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 46535 } } peers { permanent_uuid: "9a8039f9d4884aee9bc77b7e45daa7ab" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 43873 } }
I20260805 23:12:56.381023 7194 raft_consensus.cc:374] T 9988db2ba1db43509f01fb1ddc78fc25 P 162ee0db3de9468f864b397f56f554cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:12:56.381099 7194 raft_consensus.cc:729] T 9988db2ba1db43509f01fb1ddc78fc25 P 162ee0db3de9468f864b397f56f554cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 162ee0db3de9468f864b397f56f554cd, State: Initialized, Role: FOLLOWER
I20260805 23:12:56.381723 7194 consensus_queue.cc:260] T 9988db2ba1db43509f01fb1ddc78fc25 P 162ee0db3de9468f864b397f56f554cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "162ee0db3de9468f864b397f56f554cd" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 39189 } } peers { permanent_uuid: "e4c6ea201d564bdc88361eb96db36bd7" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 46535 } } peers { permanent_uuid: "9a8039f9d4884aee9bc77b7e45daa7ab" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 43873 } }
I20260805 23:12:56.385821 7197 tablet_bootstrap.cc:492] T 9988db2ba1db43509f01fb1ddc78fc25 P e4c6ea201d564bdc88361eb96db36bd7: Bootstrap starting.
I20260805 23:12:56.386543 7194 ts_tablet_manager.cc:1428] T 9988db2ba1db43509f01fb1ddc78fc25 P 162ee0db3de9468f864b397f56f554cd: Time spent starting tablet: real 0.012s user 0.006s sys 0.001s
I20260805 23:12:56.386986 7025 heartbeater.cc:503] Master 127.25.92.254:44315 was elected leader, sending a full tablet report...
I20260805 23:12:56.389403 7197 tablet_bootstrap.cc:654] T 9988db2ba1db43509f01fb1ddc78fc25 P e4c6ea201d564bdc88361eb96db36bd7: Neither blocks nor log segments found. Creating new log.
I20260805 23:12:56.390650 7197 log.cc:824] T 9988db2ba1db43509f01fb1ddc78fc25 P e4c6ea201d564bdc88361eb96db36bd7: Log is configured to *not* fsync() on all Append() calls
I20260805 23:12:56.393232 7197 tablet_bootstrap.cc:492] T 9988db2ba1db43509f01fb1ddc78fc25 P e4c6ea201d564bdc88361eb96db36bd7: No bootstrap required, opened a new log
I20260805 23:12:56.393554 7197 ts_tablet_manager.cc:1397] T 9988db2ba1db43509f01fb1ddc78fc25 P e4c6ea201d564bdc88361eb96db36bd7: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260805 23:12:56.397877 7197 raft_consensus.cc:348] T 9988db2ba1db43509f01fb1ddc78fc25 P e4c6ea201d564bdc88361eb96db36bd7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "162ee0db3de9468f864b397f56f554cd" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 39189 } } peers { permanent_uuid: "e4c6ea201d564bdc88361eb96db36bd7" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 46535 } } peers { permanent_uuid: "9a8039f9d4884aee9bc77b7e45daa7ab" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 43873 } }
I20260805 23:12:56.398198 7197 raft_consensus.cc:374] T 9988db2ba1db43509f01fb1ddc78fc25 P e4c6ea201d564bdc88361eb96db36bd7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:12:56.398262 7197 raft_consensus.cc:729] T 9988db2ba1db43509f01fb1ddc78fc25 P e4c6ea201d564bdc88361eb96db36bd7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4c6ea201d564bdc88361eb96db36bd7, State: Initialized, Role: FOLLOWER
I20260805 23:12:56.398876 7197 consensus_queue.cc:260] T 9988db2ba1db43509f01fb1ddc78fc25 P e4c6ea201d564bdc88361eb96db36bd7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "162ee0db3de9468f864b397f56f554cd" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 39189 } } peers { permanent_uuid: "e4c6ea201d564bdc88361eb96db36bd7" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 46535 } } peers { permanent_uuid: "9a8039f9d4884aee9bc77b7e45daa7ab" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 43873 } }
I20260805 23:12:56.401089 7197 ts_tablet_manager.cc:1428] T 9988db2ba1db43509f01fb1ddc78fc25 P e4c6ea201d564bdc88361eb96db36bd7: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260805 23:12:56.401540 7167 heartbeater.cc:503] Master 127.25.92.254:44315 was elected leader, sending a full tablet report...
W20260805 23:12:56.451049 6887 tablet.cc:2369] T 9988db2ba1db43509f01fb1ddc78fc25 P 9a8039f9d4884aee9bc77b7e45daa7ab: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260805 23:12:56.452239 7199 raft_consensus.cc:482] T 9988db2ba1db43509f01fb1ddc78fc25 P 162ee0db3de9468f864b397f56f554cd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 23:12:56.452454 7199 raft_consensus.cc:504] T 9988db2ba1db43509f01fb1ddc78fc25 P 162ee0db3de9468f864b397f56f554cd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "162ee0db3de9468f864b397f56f554cd" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 39189 } } peers { permanent_uuid: "e4c6ea201d564bdc88361eb96db36bd7" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 46535 } } peers { permanent_uuid: "9a8039f9d4884aee9bc77b7e45daa7ab" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 43873 } }
I20260805 23:12:56.453943 7199 leader_election.cc:288] T 9988db2ba1db43509f01fb1ddc78fc25 P 162ee0db3de9468f864b397f56f554cd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e4c6ea201d564bdc88361eb96db36bd7 (127.25.92.195:46535), 9a8039f9d4884aee9bc77b7e45daa7ab (127.25.92.193:43873)
I20260805 23:12:56.459952 7190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:12:56.454829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.194:56557 (local address 127.25.92.193:43873)
0805 23:12:56.454964 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0805 23:12:56.454969 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:12:56.455240 (+ 271us) server_negotiation.cc:407] Connection header received
0805 23:12:56.455455 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:12:56.455459 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:12:56.455509 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:12:56.455573 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:12:56.456027 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:12:56.456838 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:12:56.458616 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:12:56.459213 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:12:56.459313 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:12:56.459366 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0805 23:12:56.459737 (+ 371us) server_negotiation.cc:299] Negotiation successful
0805 23:12:56.459822 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260805 23:12:56.460641 7203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:12:56.454358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.193:43873 (local address 127.25.92.194:56557)
0805 23:12:56.455144 (+ 786us) negotiation.cc:107] Waiting for socket to connect
0805 23:12:56.455159 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0805 23:12:56.455325 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:12:56.455725 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:12:56.455728 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:12:56.455745 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:12:56.455888 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:12:56.455893 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:12:56.456990 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:12:56.456993 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:12:56.458451 (+ 1458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:12:56.458462 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:12:56.459471 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:12:56.459475 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:12:56.459587 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:12:56.459607 (+ 20us) client_negotiation.cc:769] Sending connection context
0805 23:12:56.459716 (+ 109us) client_negotiation.cc:240] Negotiation successful
0805 23:12:56.460464 (+ 748us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":701,"mutex_wait_us":10,"thread_start_us":128,"threads_started":1}
I20260805 23:12:56.461463 6837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9988db2ba1db43509f01fb1ddc78fc25" candidate_uuid: "162ee0db3de9468f864b397f56f554cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a8039f9d4884aee9bc77b7e45daa7ab" is_pre_election: true
I20260805 23:12:56.461874 6837 raft_consensus.cc:2454] T 9988db2ba1db43509f01fb1ddc78fc25 P 9a8039f9d4884aee9bc77b7e45daa7ab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 162ee0db3de9468f864b397f56f554cd in term 0.
I20260805 23:12:56.462453 6908 leader_election.cc:302] T 9988db2ba1db43509f01fb1ddc78fc25 P 162ee0db3de9468f864b397f56f554cd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 162ee0db3de9468f864b397f56f554cd, 9a8039f9d4884aee9bc77b7e45daa7ab; no voters:
I20260805 23:12:56.462792 7199 raft_consensus.cc:2781] T 9988db2ba1db43509f01fb1ddc78fc25 P 162ee0db3de9468f864b397f56f554cd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260805 23:12:56.462878 7199 raft_consensus.cc:482] T 9988db2ba1db43509f01fb1ddc78fc25 P 162ee0db3de9468f864b397f56f554cd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260805 23:12:56.462929 7199 raft_consensus.cc:3037] T 9988db2ba1db43509f01fb1ddc78fc25 P 162ee0db3de9468f864b397f56f554cd [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:12:56.464404 7199 raft_consensus.cc:504] T 9988db2ba1db43509f01fb1ddc78fc25 P 162ee0db3de9468f864b397f56f554cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "162ee0db3de9468f864b397f56f554cd" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 39189 } } peers { permanent_uuid: "e4c6ea201d564bdc88361eb96db36bd7" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 46535 } } peers { permanent_uuid: "9a8039f9d4884aee9bc77b7e45daa7ab" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 43873 } }
I20260805 23:12:56.464829 7199 leader_election.cc:288] T 9988db2ba1db43509f01fb1ddc78fc25 P 162ee0db3de9468f864b397f56f554cd [CANDIDATE]: Term 1 election: Requested vote from peers e4c6ea201d564bdc88361eb96db36bd7 (127.25.92.195:46535), 9a8039f9d4884aee9bc77b7e45daa7ab (127.25.92.193:43873)
I20260805 23:12:56.465229 7191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:12:56.454891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.194:44143 (local address 127.25.92.195:46535)
0805 23:12:56.455052 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0805 23:12:56.455057 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:12:56.455384 (+ 327us) server_negotiation.cc:407] Connection header received
0805 23:12:56.455636 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:12:56.455640 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:12:56.455702 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:12:56.455766 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:12:56.456274 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:12:56.457146 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:12:56.459538 (+ 2392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:12:56.460210 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:12:56.460325 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:12:56.460384 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0805 23:12:56.464984 (+ 4600us) server_negotiation.cc:299] Negotiation successful
0805 23:12:56.465102 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260805 23:12:56.465373 6837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9988db2ba1db43509f01fb1ddc78fc25" candidate_uuid: "162ee0db3de9468f864b397f56f554cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a8039f9d4884aee9bc77b7e45daa7ab"
I20260805 23:12:56.465523 6837 raft_consensus.cc:3037] T 9988db2ba1db43509f01fb1ddc78fc25 P 9a8039f9d4884aee9bc77b7e45daa7ab [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:12:56.465670 7202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:12:56.454412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.195:46535 (local address 127.25.92.194:44143)
0805 23:12:56.455277 (+ 865us) negotiation.cc:107] Waiting for socket to connect
0805 23:12:56.455295 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0805 23:12:56.455474 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:12:56.455933 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:12:56.455937 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:12:56.455956 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:12:56.456117 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:12:56.456125 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:12:56.457323 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:12:56.457327 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:12:56.458735 (+ 1408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:12:56.458748 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:12:56.464671 (+ 5923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:12:56.464674 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:12:56.464798 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:12:56.464830 (+ 32us) client_negotiation.cc:769] Sending connection context
0805 23:12:56.465377 (+ 547us) client_negotiation.cc:240] Negotiation successful
0805 23:12:56.465469 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":788,"mutex_wait_us":2,"thread_start_us":114,"threads_started":1}
I20260805 23:12:56.466516 7102 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9988db2ba1db43509f01fb1ddc78fc25" candidate_uuid: "162ee0db3de9468f864b397f56f554cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4c6ea201d564bdc88361eb96db36bd7" is_pre_election: true
I20260805 23:12:56.466845 7102 raft_consensus.cc:2454] T 9988db2ba1db43509f01fb1ddc78fc25 P e4c6ea201d564bdc88361eb96db36bd7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 162ee0db3de9468f864b397f56f554cd in term 0.
I20260805 23:12:56.466899 6837 raft_consensus.cc:2454] T 9988db2ba1db43509f01fb1ddc78fc25 P 9a8039f9d4884aee9bc77b7e45daa7ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 162ee0db3de9468f864b397f56f554cd in term 1.
I20260805 23:12:56.467156 7120 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9988db2ba1db43509f01fb1ddc78fc25" candidate_uuid: "162ee0db3de9468f864b397f56f554cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4c6ea201d564bdc88361eb96db36bd7"
I20260805 23:12:56.467275 7120 raft_consensus.cc:3037] T 9988db2ba1db43509f01fb1ddc78fc25 P e4c6ea201d564bdc88361eb96db36bd7 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:12:56.467706 6908 leader_election.cc:302] T 9988db2ba1db43509f01fb1ddc78fc25 P 162ee0db3de9468f864b397f56f554cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 162ee0db3de9468f864b397f56f554cd, 9a8039f9d4884aee9bc77b7e45daa7ab; no voters:
I20260805 23:12:56.467888 7199 raft_consensus.cc:2781] T 9988db2ba1db43509f01fb1ddc78fc25 P 162ee0db3de9468f864b397f56f554cd [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:12:56.468115 7199 raft_consensus.cc:686] T 9988db2ba1db43509f01fb1ddc78fc25 P 162ee0db3de9468f864b397f56f554cd [term 1 LEADER]: Becoming Leader. State: Replica: 162ee0db3de9468f864b397f56f554cd, State: Running, Role: LEADER
I20260805 23:12:56.468482 7199 consensus_queue.cc:237] T 9988db2ba1db43509f01fb1ddc78fc25 P 162ee0db3de9468f864b397f56f554cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "162ee0db3de9468f864b397f56f554cd" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 39189 } } peers { permanent_uuid: "e4c6ea201d564bdc88361eb96db36bd7" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 46535 } } peers { permanent_uuid: "9a8039f9d4884aee9bc77b7e45daa7ab" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 43873 } }
I20260805 23:12:56.468626 7120 raft_consensus.cc:2454] T 9988db2ba1db43509f01fb1ddc78fc25 P e4c6ea201d564bdc88361eb96db36bd7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 162ee0db3de9468f864b397f56f554cd in term 1.
I20260805 23:12:56.472193 6678 catalog_manager.cc:5495] T 9988db2ba1db43509f01fb1ddc78fc25 P 162ee0db3de9468f864b397f56f554cd reported cstate change: term changed from 0 to 1, leader changed from <none> to 162ee0db3de9468f864b397f56f554cd (127.25.92.194). New cstate: current_term: 1 leader_uuid: "162ee0db3de9468f864b397f56f554cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "162ee0db3de9468f864b397f56f554cd" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 39189 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e4c6ea201d564bdc88361eb96db36bd7" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 46535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a8039f9d4884aee9bc77b7e45daa7ab" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 43873 } health_report { overall_health: UNKNOWN } } }
I20260805 23:12:56.502357 7192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:12:56.497556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54662 (local address 127.25.92.194:39189)
0805 23:12:56.497712 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0805 23:12:56.497717 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:12:56.497734 (+ 17us) server_negotiation.cc:407] Connection header received
0805 23:12:56.497831 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:12:56.497834 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:12:56.497901 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:12:56.498022 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 23:12:56.498351 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:12:56.499180 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:12:56.500570 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:12:56.500827 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:12:56.500990 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:12:56.501349 (+ 359us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 23:12:56.501835 (+ 486us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 23:12:56.501933 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0805 23:12:56.502061 (+ 128us) server_negotiation.cc:299] Negotiation successful
0805 23:12:56.502129 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260805 23:12:56.509584 7120 raft_consensus.cc:1261] T 9988db2ba1db43509f01fb1ddc78fc25 P e4c6ea201d564bdc88361eb96db36bd7 [term 1 FOLLOWER]: Refusing update from remote peer 162ee0db3de9468f864b397f56f554cd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:12:56.509581 6837 raft_consensus.cc:1261] T 9988db2ba1db43509f01fb1ddc78fc25 P 9a8039f9d4884aee9bc77b7e45daa7ab [term 1 FOLLOWER]: Refusing update from remote peer 162ee0db3de9468f864b397f56f554cd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:12:56.510437 7199 consensus_queue.cc:1035] T 9988db2ba1db43509f01fb1ddc78fc25 P 162ee0db3de9468f864b397f56f554cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "e4c6ea201d564bdc88361eb96db36bd7" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 46535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:12:56.510715 7204 consensus_queue.cc:1035] T 9988db2ba1db43509f01fb1ddc78fc25 P 162ee0db3de9468f864b397f56f554cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a8039f9d4884aee9bc77b7e45daa7ab" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 43873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260805 23:12:56.518229 7026 tablet.cc:2369] T 9988db2ba1db43509f01fb1ddc78fc25 P 162ee0db3de9468f864b397f56f554cd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260805 23:12:56.520274 7207 mvcc.cc:205] Tried to move back new op lower bound from 7315339577374810112 to 7315339577218576384. Current Snapshot: MvccSnapshot[applied={T|T < 7315339577374810112}]
I20260805 23:12:56.529556 7209 mvcc.cc:205] Tried to move back new op lower bound from 7315339577374810112 to 7315339577218576384. Current Snapshot: MvccSnapshot[applied={T|T < 7315339577374810112}]
I20260805 23:12:56.526396 7208 mvcc.cc:205] Tried to move back new op lower bound from 7315339577374810112 to 7315339577218576384. Current Snapshot: MvccSnapshot[applied={T|T < 7315339577374810112}]
I20260805 23:12:56.530447 6678 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58126:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:12:56.530668 6678 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58126:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:12:56.533682 6678 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ba0742b645c94c13b3db716954ead8aa: Sending DeleteTablet for 3 replicas of tablet 9988db2ba1db43509f01fb1ddc78fc25
I20260805 23:12:56.534482 6956 tablet_service.cc:1514] Processing DeleteTablet for tablet 9988db2ba1db43509f01fb1ddc78fc25 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:12:56 UTC) from {username='kudu', principal='kudu/127.25.92.254@KRBTEST.COM'} at 127.0.0.1:54650
I20260805 23:12:56.535426 7100 tablet_service.cc:1514] Processing DeleteTablet for tablet 9988db2ba1db43509f01fb1ddc78fc25 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:12:56 UTC) from {username='kudu', principal='kudu/127.25.92.254@KRBTEST.COM'} at 127.0.0.1:39716
I20260805 23:12:56.535861 6815 tablet_service.cc:1514] Processing DeleteTablet for tablet 9988db2ba1db43509f01fb1ddc78fc25 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:12:56 UTC) from {username='kudu', principal='kudu/127.25.92.254@KRBTEST.COM'} at 127.0.0.1:44648
I20260805 23:12:56.537271 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 6717
I20260805 23:12:56.543409 7218 tablet_replica.cc:330] T 9988db2ba1db43509f01fb1ddc78fc25 P 162ee0db3de9468f864b397f56f554cd: stopping tablet replica
I20260805 23:12:56.543787 7218 raft_consensus.cc:2229] T 9988db2ba1db43509f01fb1ddc78fc25 P 162ee0db3de9468f864b397f56f554cd [term 1 LEADER]: Raft consensus shutting down.
I20260805 23:12:56.544283 7218 raft_consensus.cc:2258] T 9988db2ba1db43509f01fb1ddc78fc25 P 162ee0db3de9468f864b397f56f554cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:12:56.546660 7218 ts_tablet_manager.cc:1905] T 9988db2ba1db43509f01fb1ddc78fc25 P 162ee0db3de9468f864b397f56f554cd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:12:56.549731 7220 tablet_replica.cc:330] T 9988db2ba1db43509f01fb1ddc78fc25 P e4c6ea201d564bdc88361eb96db36bd7: stopping tablet replica
I20260805 23:12:56.550053 7220 raft_consensus.cc:2229] T 9988db2ba1db43509f01fb1ddc78fc25 P e4c6ea201d564bdc88361eb96db36bd7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 23:12:56.550380 7220 raft_consensus.cc:2258] T 9988db2ba1db43509f01fb1ddc78fc25 P e4c6ea201d564bdc88361eb96db36bd7 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260805 23:12:56.552802 6661 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.25.92.193:43873 (error 108)
I20260805 23:12:56.554425 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 6889
I20260805 23:12:56.554452 7188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:12:56.554153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.193:43873 (local address 127.0.0.1:44660)
0805 23:12:56.554273 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0805 23:12:56.554359 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.193:43873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260805 23:12:56.554629 6661 catalog_manager.cc:4584] TS 9a8039f9d4884aee9bc77b7e45daa7ab (127.25.92.193:43873): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9988db2ba1db43509f01fb1ddc78fc25: Network error: Client connection negotiation failed: client connection to 127.25.92.193:43873: connect: Connection refused (error 111)
I20260805 23:12:56.555404 7220 ts_tablet_manager.cc:1905] T 9988db2ba1db43509f01fb1ddc78fc25 P e4c6ea201d564bdc88361eb96db36bd7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:12:56.559167 7220 ts_tablet_manager.cc:1918] T 9988db2ba1db43509f01fb1ddc78fc25 P e4c6ea201d564bdc88361eb96db36bd7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 23:12:56.559315 7220 log.cc:1196] T 9988db2ba1db43509f01fb1ddc78fc25 P e4c6ea201d564bdc88361eb96db36bd7: Deleting WAL directory at /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971406251336-25971-0/minicluster-data/ts-2/wal/wals/9988db2ba1db43509f01fb1ddc78fc25
I20260805 23:12:56.559926 7220 ts_tablet_manager.cc:1939] T 9988db2ba1db43509f01fb1ddc78fc25 P e4c6ea201d564bdc88361eb96db36bd7: Deleting consensus metadata
I20260805 23:12:56.561276 6664 catalog_manager.cc:4841] TS e4c6ea201d564bdc88361eb96db36bd7 (127.25.92.195:46535): tablet 9988db2ba1db43509f01fb1ddc78fc25 (table test-table [id=adf01a54dcb04783a9b1625d2d12d05e]) successfully deleted
I20260805 23:12:56.566094 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 7028
I20260805 23:12:56.570487 7188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:12:56.570128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.194:39189 (local address 127.0.0.1:54664)
0805 23:12:56.570299 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0805 23:12:56.570375 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.194:39189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260805 23:12:56.576632 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 6644
I20260805 23:12:56.585709 25971 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260805 23:12:56.931802 25971 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260805 23:12:56.931957 25971 mini_ranger.cc:66] Stopping Ranger...
I20260805 23:12:57.278271 25971 mini_ranger.cc:68] Stopped Ranger
2026-08-05 23:12:57.278 UTC [5655] LOG: received smart shutdown request
2026-08-05 23:12:57.284 UTC [5655] LOG: background worker "logical replication launcher" (PID 5664) exited with exit code 1
2026-08-05 23:12:57.284 UTC [5659] LOG: shutting down
2026-08-05 23:12:57.513 UTC [5655] LOG: database system is shut down
2026-08-05T23:12:57Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (52932 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971406251336-25971-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 23:12:57 dist-test-slave-ss4n krb5kdc[7232](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 23:12:57 dist-test-slave-ss4n krb5kdc[7232](info): set up 2 sockets
Aug 05 23:12:57 dist-test-slave-ss4n krb5kdc[7232](info): commencing operation
krb5kdc: starting...
W20260805 23:12:58.734318 25971 mini_kdc.cc:121] Time spent starting KDC: real 1.070s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971406251336-25971-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971406251336-25971-0/krb5kdc/test-user.keytab.
Aug 05 23:12:58 dist-test-slave-ss4n krb5kdc[7232](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971578, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T23:12:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:12:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.92.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.92.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.92.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab.
Entry for principal kudu/127.25.92.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab.
I20260805 23:12:58.882084 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971406251336-25971-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971406251336-25971-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971406251336-25971-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.254:46837
--webserver_interface=127.25.92.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab
--principal=kudu/127.25.92.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:33525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.92.254:46837,127.25.92.253:37205,127.25.92.252:38015
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:12:59.008831 7246 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:12:59.009155 7246 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:12:59.009197 7246 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:12:59.013301 7246 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:12:59.013389 7246 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:12:59.013409 7246 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:12:59.013427 7246 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:12:59.013441 7246 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:12:59.019429 7246 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:33525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971406251336-25971-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971406251336-25971-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.92.254:46837,127.25.92.253:37205,127.25.92.252:38015
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab
--principal=kudu/127.25.92.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.254:46837
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.92.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7246
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971406251336-25971-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:12:59.020587 7246 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:12:59.021651 7246 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:12:59.030987 7252 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:12:59.030988 7254 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:12:59.031191 7246 server_base.cc:1034] running on GCE node
W20260805 23:12:59.030988 7251 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:12:59.032042 7246 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:12:59.033183 7246 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:12:59.034364 7246 hybrid_clock.cc:648] HybridClock initialized: now 1785971579034334 us; error 45 us; skew 500 ppm
Aug 05 23:12:59 dist-test-slave-ss4n krb5kdc[7232](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971579, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:12:59.037724 7246 init.cc:377] Logged in from keytab as kudu/127.25.92.254@KRBTEST.COM (short username kudu)
I20260805 23:12:59.038846 7246 webserver.cc:460] Webserver started at http://127.25.92.254:37173/ using document root <none> and password file <none>
I20260805 23:12:59.039399 7246 fs_manager.cc:359] Metadata directory not provided
I20260805 23:12:59.039475 7246 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:12:59.039656 7246 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:12:59.041914 7246 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971406251336-25971-0/minicluster-data/master-0/data/instance:
uuid: "0b1e8f104b67405f97ccc1d3c7335629"
format_stamp: "Formatted at 2026-08-05 23:12:59 on dist-test-slave-ss4n"
server_key: "2cf7707ed9e2ffafe88a364d251db30a"
server_key_iv: "5a9d78f7ffb6e07333070dcbf2583451"
server_key_version: "encryptionkey@0"
I20260805 23:12:59.042531 7246 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971406251336-25971-0/minicluster-data/master-0/wal/instance:
uuid: "0b1e8f104b67405f97ccc1d3c7335629"
format_stamp: "Formatted at 2026-08-05 23:12:59 on dist-test-slave-ss4n"
server_key: "2cf7707ed9e2ffafe88a364d251db30a"
server_key_iv: "5a9d78f7ffb6e07333070dcbf2583451"
server_key_version: "encryptionkey@0"
I20260805 23:12:59.046737 7246 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260805 23:12:59.049382 7262 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:12:59.050897 7246 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260805 23:12:59.051086 7246 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971406251336-25971-0/minicluster-data/master-0/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971406251336-25971-0/minicluster-data/master-0/wal
uuid: "0b1e8f104b67405f97ccc1d3c7335629"
format_stamp: "Formatted at 2026-08-05 23:12:59 on dist-test-slave-ss4n"
server_key: "2cf7707ed9e2ffafe88a364d251db30a"
server_key_iv: "5a9d78f7ffb6e07333070dcbf2583451"
server_key_version: "encryptionkey@0"
I20260805 23:12:59.051227 7246 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971406251336-25971-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971406251336-25971-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971406251336-25971-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:12:59.074133 7246 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:12:59.077337 7246 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:12:59.077630 7246 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:12:59.085754 7246 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.254:46837
I20260805 23:12:59.085817 7319 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.254:46837 every 8 connection(s)
I20260805 23:12:59.086803 7246 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
I20260805 23:12:59.089908 7320 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:12:59.090626 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 7246
I20260805 23:12:59.090777 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971406251336-25971-0/minicluster-data/master-0/wal/instance
I20260805 23:12:59.091096 25971 external_mini_cluster.cc:1428] Setting key 06dd5a54f3c8d585c2a01c670f379920
I20260805 23:12:59.094063 7320 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.92.254" port: 46837 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.25.92.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.92.253@KRBTEST.COM" created.
Aug 05 23:12:59 dist-test-slave-ss4n krb5kdc[7232](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971579, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.254@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM
I20260805 23:12:59.110894 7325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:12:59.097853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55730 (local address 127.25.92.254:46837)
0805 23:12:59.098171 (+ 318us) server_negotiation.cc:206] Beginning negotiation
0805 23:12:59.098181 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0805 23:12:59.098235 (+ 54us) server_negotiation.cc:407] Connection header received
0805 23:12:59.098747 (+ 512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:12:59.098752 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:12:59.098928 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:12:59.099111 (+ 183us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:12:59.100030 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:12:59.101008 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:12:59.102199 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:12:59.102685 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:12:59.102848 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:12:59.105634 (+ 2786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:12:59.105641 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:12:59.105652 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:12:59.105675 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:12:59.107464 (+ 1789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:12:59.108132 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:12:59.108137 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:12:59.108138 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:12:59.108246 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:12:59.108671 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:12:59.108674 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:12:59.108675 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:12:59.109078 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:12:59.109181 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0805 23:12:59.109780 (+ 599us) server_negotiation.cc:299] Negotiation successful
0805 23:12:59.110008 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":6,"server-negotiator.queue_time_us":217,"thread_start_us":114,"threads_started":1}
I20260805 23:12:59.110898 7324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:12:59.096689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:46837 (local address 127.0.0.1:55730)
0805 23:12:59.097300 (+ 611us) negotiation.cc:107] Waiting for socket to connect
0805 23:12:59.097346 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0805 23:12:59.098435 (+ 1089us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:12:59.099200 (+ 765us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:12:59.099206 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:12:59.099629 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:12:59.099877 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:12:59.099890 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:12:59.101150 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:12:59.101153 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:12:59.102064 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:12:59.102074 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:12:59.102881 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:12:59.102889 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:12:59.103075 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:12:59.103783 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:12:59.103801 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:12:59.105408 (+ 1607us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:12:59.107587 (+ 2179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:12:59.107600 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:12:59.107613 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:12:59.107985 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:12:59.108353 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:12:59.108358 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:12:59.108363 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:12:59.108552 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:12:59.109213 (+ 661us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:12:59.109223 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:12:59.109420 (+ 197us) client_negotiation.cc:769] Sending connection context
0805 23:12:59.109680 (+ 260us) client_negotiation.cc:240] Negotiation successful
0805 23:12:59.110132 (+ 452us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":405,"mutex_wait_us":89,"thread_start_us":164,"threads_started":1}
I20260805 23:12:59.113853 7320 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.92.253" port: 37205 } has no permanent_uuid. Determining permanent_uuid...
I20260805 23:12:59.114888 7324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:12:59.114434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.253:37205 (local address 127.0.0.1:45900)
0805 23:12:59.114687 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0805 23:12:59.114786 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.253:37205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154}
W20260805 23:12:59.115145 7266 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.25.92.253:37205: connect: Connection refused (error 111)
I20260805 23:12:59.116124 7324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:12:59.115887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.253:37205 (local address 127.0.0.1:45912)
0805 23:12:59.116003 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0805 23:12:59.116056 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.253:37205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260805 23:12:59.116307 7320 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.92.253:37205: Network error: Client connection negotiation failed: client connection to 127.25.92.253:37205: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.92.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.253.keytab.
Entry for principal kudu/127.25.92.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.253.keytab.
I20260805 23:12:59.122045 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971406251336-25971-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971406251336-25971-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971406251336-25971-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971406251336-25971-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.253:37205
--webserver_interface=127.25.92.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.253.keytab
--principal=kudu/127.25.92.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:33525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.92.254:46837,127.25.92.253:37205,127.25.92.252:38015
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260805 23:12:59.165611 7320 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.92.253" port: 37205 } attempt: 1
I20260805 23:12:59.166700 7324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:12:59.166284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.253:37205 (local address 127.0.0.1:45926)
0805 23:12:59.166514 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0805 23:12:59.166603 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.253:37205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260805 23:12:59.167475 7324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:12:59.167248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.253:37205 (local address 127.0.0.1:45934)
0805 23:12:59.167360 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0805 23:12:59.167414 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.253:37205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260805 23:12:59.167640 7320 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.92.253:37205: Network error: Client connection negotiation failed: client connection to 127.25.92.253:37205: connect: Connection refused (error 111)
I20260805 23:12:59.235857 7320 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.92.253" port: 37205 } attempt: 2
I20260805 23:12:59.236814 7324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:12:59.236455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.253:37205 (local address 127.0.0.1:45950)
0805 23:12:59.236641 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0805 23:12:59.236717 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.253:37205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260805 23:12:59.237767 7324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:12:59.237385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.253:37205 (local address 127.0.0.1:45966)
0805 23:12:59.237653 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0805 23:12:59.237708 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.253:37205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260805 23:12:59.237955 7320 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.92.253:37205: Network error: Client connection negotiation failed: client connection to 127.25.92.253:37205: connect: Connection refused (error 111)
W20260805 23:12:59.238245 7328 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:12:59.238508 7328 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:12:59.238550 7328 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:12:59.242837 7328 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:12:59.242913 7328 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:12:59.242933 7328 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:12:59.242952 7328 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:12:59.242967 7328 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:12:59.248631 7328 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:33525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971406251336-25971-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971406251336-25971-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.25.92.254:46837,127.25.92.253:37205,127.25.92.252:38015
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.253.keytab
--principal=kudu/127.25.92.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.253:37205
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971406251336-25971-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.25.92.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7328
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971406251336-25971-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:12:59.249832 7328 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:12:59.250836 7328 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:12:59.260033 7335 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:12:59.260033 7337 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:12:59.259985 7333 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:12:59.260443 7328 server_base.cc:1034] running on GCE node
I20260805 23:12:59.260969 7328 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:12:59.262277 7328 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:12:59.263450 7328 hybrid_clock.cc:648] HybridClock initialized: now 1785971579263437 us; error 35 us; skew 500 ppm
Aug 05 23:12:59 dist-test-slave-ss4n krb5kdc[7232](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971579, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:12:59.266561 7328 init.cc:377] Logged in from keytab as kudu/127.25.92.253@KRBTEST.COM (short username kudu)
I20260805 23:12:59.267565 7328 webserver.cc:460] Webserver started at http://127.25.92.253:43531/ using document root <none> and password file <none>
I20260805 23:12:59.268124 7328 fs_manager.cc:359] Metadata directory not provided
I20260805 23:12:59.268193 7328 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:12:59.268404 7328 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:12:59.270702 7328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971406251336-25971-0/minicluster-data/master-1/data/instance:
uuid: "802657d49388472a8de98cd7b46e2361"
format_stamp: "Formatted at 2026-08-05 23:12:59 on dist-test-slave-ss4n"
server_key: "6d4d5897d8669c48717a457b6b397430"
server_key_iv: "ac7930c986160100b10b40cb315102d4"
server_key_version: "encryptionkey@0"
I20260805 23:12:59.271275 7328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971406251336-25971-0/minicluster-data/master-1/wal/instance:
uuid: "802657d49388472a8de98cd7b46e2361"
format_stamp: "Formatted at 2026-08-05 23:12:59 on dist-test-slave-ss4n"
server_key: "6d4d5897d8669c48717a457b6b397430"
server_key_iv: "ac7930c986160100b10b40cb315102d4"
server_key_version: "encryptionkey@0"
I20260805 23:12:59.275348 7328 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260805 23:12:59.278162 7345 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:12:59.279721 7328 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:12:59.279840 7328 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971406251336-25971-0/minicluster-data/master-1/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971406251336-25971-0/minicluster-data/master-1/wal
uuid: "802657d49388472a8de98cd7b46e2361"
format_stamp: "Formatted at 2026-08-05 23:12:59 on dist-test-slave-ss4n"
server_key: "6d4d5897d8669c48717a457b6b397430"
server_key_iv: "ac7930c986160100b10b40cb315102d4"
server_key_version: "encryptionkey@0"
I20260805 23:12:59.279945 7328 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971406251336-25971-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971406251336-25971-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971406251336-25971-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:12:59.297317 7328 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:12:59.301091 7328 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:12:59.301285 7328 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:12:59.309384 7328 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.253:37205
I20260805 23:12:59.309391 7401 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.253:37205 every 8 connection(s)
I20260805 23:12:59.310489 7328 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971406251336-25971-0/minicluster-data/master-1/data/info.pb
I20260805 23:12:59.313340 7403 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:12:59.317299 7403 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.92.254" port: 46837 } has no permanent_uuid. Determining permanent_uuid...
I20260805 23:12:59.319638 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 7328
I20260805 23:12:59.319777 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971406251336-25971-0/minicluster-data/master-1/wal/instance
I20260805 23:12:59.320039 25971 external_mini_cluster.cc:1428] Setting key 476772bdf24cb6625b506f5141135e1a
WARNING: no policy specified for kudu/127.25.92.252@KRBTEST.COM; defaulting to no policy
Aug 05 23:12:59 dist-test-slave-ss4n krb5kdc[7232](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971579, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.253@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM
I20260805 23:12:59.329155 7320 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.92.253" port: 37205 } attempt: 3
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.92.252@KRBTEST.COM" created.
I20260805 23:12:59.333791 7325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:12:59.319534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55746 (local address 127.25.92.254:46837)
0805 23:12:59.319725 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0805 23:12:59.319732 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0805 23:12:59.320771 (+ 1039us) server_negotiation.cc:407] Connection header received
0805 23:12:59.321837 (+ 1066us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:12:59.321842 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:12:59.321915 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:12:59.322033 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:12:59.323349 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:12:59.324232 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:12:59.325836 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:12:59.326301 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:12:59.326432 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:12:59.329263 (+ 2831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:12:59.329288 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:12:59.329293 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:12:59.329327 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:12:59.331324 (+ 1997us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:12:59.331967 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:12:59.331973 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:12:59.331975 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:12:59.332067 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:12:59.332489 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:12:59.332493 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:12:59.332495 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:12:59.332749 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:12:59.332870 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0805 23:12:59.333441 (+ 571us) server_negotiation.cc:299] Negotiation successful
0805 23:12:59.333613 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260805 23:12:59.334970 7405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:12:59.320014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:46837 (local address 127.0.0.1:55746)
0805 23:12:59.320583 (+ 569us) negotiation.cc:107] Waiting for socket to connect
0805 23:12:59.320638 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0805 23:12:59.321608 (+ 970us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:12:59.322257 (+ 649us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:12:59.322264 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:12:59.322640 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:12:59.323177 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:12:59.323191 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:12:59.324446 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:12:59.324453 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:12:59.325688 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:12:59.325701 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:12:59.326485 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:12:59.326489 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:12:59.326685 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:12:59.327445 (+ 760us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:12:59.327464 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:12:59.329028 (+ 1564us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:12:59.331457 (+ 2429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:12:59.331464 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:12:59.331485 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:12:59.331855 (+ 370us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:12:59.332168 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:12:59.332172 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:12:59.332174 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:12:59.332375 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:12:59.332870 (+ 495us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:12:59.332879 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:12:59.333165 (+ 286us) client_negotiation.cc:769] Sending connection context
0805 23:12:59.333382 (+ 217us) client_negotiation.cc:240] Negotiation successful
0805 23:12:59.333795 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":160,"threads_started":1}
I20260805 23:12:59.336784 7403 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.92.253" port: 37205 } has no permanent_uuid. Determining permanent_uuid...
Aug 05 23:12:59 dist-test-slave-ss4n krb5kdc[7232](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971579, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.254@KRBTEST.COM for kudu/127.25.92.253@KRBTEST.COM
I20260805 23:12:59.342752 7324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:12:59.329733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.253:37205 (local address 127.0.0.1:45972)
0805 23:12:59.329878 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0805 23:12:59.329897 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0805 23:12:59.329987 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:12:59.330863 (+ 876us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:12:59.330866 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:12:59.331259 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:12:59.331381 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:12:59.331387 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:12:59.332652 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:12:59.332659 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:12:59.333588 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:12:59.333603 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:12:59.334506 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:12:59.334510 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:12:59.334631 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:12:59.335411 (+ 780us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:12:59.335423 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:12:59.337937 (+ 2514us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:12:59.340601 (+ 2664us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:12:59.340605 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:12:59.340610 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:12:59.340905 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:12:59.341276 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:12:59.341280 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:12:59.341283 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:12:59.341408 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:12:59.342101 (+ 693us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:12:59.342105 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:12:59.342248 (+ 143us) client_negotiation.cc:769] Sending connection context
0805 23:12:59.342370 (+ 122us) client_negotiation.cc:240] Negotiation successful
0805 23:12:59.342551 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260805 23:12:59.343042 7407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:12:59.329926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45972 (local address 127.25.92.253:37205)
0805 23:12:59.330332 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0805 23:12:59.330344 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0805 23:12:59.330366 (+ 22us) server_negotiation.cc:407] Connection header received
0805 23:12:59.330406 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:12:59.330410 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:12:59.330704 (+ 294us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:12:59.330880 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:12:59.331528 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:12:59.332512 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:12:59.333737 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:12:59.334373 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:12:59.334480 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:12:59.338172 (+ 3692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:12:59.338183 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:12:59.338198 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:12:59.338223 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:12:59.340446 (+ 2223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:12:59.341031 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:12:59.341038 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:12:59.341041 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:12:59.341149 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:12:59.341556 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:12:59.341561 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:12:59.341562 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:12:59.341956 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:12:59.342130 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0805 23:12:59.342491 (+ 361us) server_negotiation.cc:299] Negotiation successful
0805 23:12:59.342654 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269,"spinlock_wait_cycles":768,"thread_start_us":138,"threads_started":1}
Aug 05 23:12:59 dist-test-slave-ss4n krb5kdc[7232](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971579, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.253@KRBTEST.COM for kudu/127.25.92.253@KRBTEST.COM
I20260805 23:12:59.345791 7320 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.92.252" port: 38015 } has no permanent_uuid. Determining permanent_uuid...
I20260805 23:12:59.346686 7324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:12:59.346396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.252:38015 (local address 127.0.0.1:38716)
0805 23:12:59.346516 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0805 23:12:59.346572 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.252:38015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260805 23:12:59.347509 7324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:12:59.347187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.252:38015 (local address 127.0.0.1:38726)
0805 23:12:59.347402 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0805 23:12:59.347446 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.252:38015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131}
W20260805 23:12:59.347674 7320 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.92.252:38015: Network error: Client connection negotiation failed: client connection to 127.25.92.252:38015: connect: Connection refused (error 111)
I20260805 23:12:59.348177 7405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:12:59.337454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.253:37205 (local address 127.0.0.1:45988)
0805 23:12:59.337629 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0805 23:12:59.337647 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0805 23:12:59.337735 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:12:59.338384 (+ 649us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:12:59.338390 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:12:59.338588 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:12:59.338745 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:12:59.338757 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:12:59.340063 (+ 1306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:12:59.340070 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:12:59.340648 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:12:59.340656 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:12:59.341256 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:12:59.341260 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:12:59.341393 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:12:59.341893 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:12:59.341912 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:12:59.343424 (+ 1512us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:12:59.345750 (+ 2326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:12:59.345757 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:12:59.345762 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:12:59.346024 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:12:59.346366 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:12:59.346368 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:12:59.346369 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:12:59.346525 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:12:59.347324 (+ 799us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:12:59.347327 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:12:59.347585 (+ 258us) client_negotiation.cc:769] Sending connection context
0805 23:12:59.347793 (+ 208us) client_negotiation.cc:240] Negotiation successful
0805 23:12:59.347983 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":85}
I20260805 23:12:59.348251 7409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:12:59.337692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45988 (local address 127.25.92.253:37205)
0805 23:12:59.338136 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0805 23:12:59.338147 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0805 23:12:59.338161 (+ 14us) server_negotiation.cc:407] Connection header received
0805 23:12:59.338191 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:12:59.338194 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:12:59.338238 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:12:59.338339 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:12:59.339239 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:12:59.339930 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:12:59.340778 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:12:59.341118 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:12:59.341247 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:12:59.343587 (+ 2340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:12:59.343591 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:12:59.343594 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:12:59.343619 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:12:59.345624 (+ 2005us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:12:59.346121 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:12:59.346123 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:12:59.346124 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:12:59.346229 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:12:59.346705 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:12:59.346709 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:12:59.346711 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:12:59.347133 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:12:59.347240 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0805 23:12:59.347882 (+ 642us) server_negotiation.cc:299] Negotiation successful
0805 23:12:59.348044 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":214,"threads_started":1}
I20260805 23:12:59.349099 7403 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.92.252" port: 38015 } has no permanent_uuid. Determining permanent_uuid...
I20260805 23:12:59.349901 7405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:12:59.349624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.252:38015 (local address 127.0.0.1:38742)
0805 23:12:59.349745 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0805 23:12:59.349838 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.252:38015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260805 23:12:59.350139 7346 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.25.92.252:38015: connect: Connection refused (error 111)
I20260805 23:12:59.351119 7405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:12:59.350889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.252:38015 (local address 127.0.0.1:38748)
0805 23:12:59.351005 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0805 23:12:59.351064 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.252:38015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260805 23:12:59.351358 7403 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.92.252:38015: Network error: Client connection negotiation failed: client connection to 127.25.92.252:38015: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.92.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.252.keytab.
Entry for principal kudu/127.25.92.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.252.keytab.
I20260805 23:12:59.354265 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971406251336-25971-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971406251336-25971-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971406251336-25971-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971406251336-25971-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.252:38015
--webserver_interface=127.25.92.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.252.keytab
--principal=kudu/127.25.92.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:33525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.92.254:46837,127.25.92.253:37205,127.25.92.252:38015
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260805 23:12:59.378844 7320 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.92.252" port: 38015 } attempt: 1
I20260805 23:12:59.379768 7324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:12:59.379456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.252:38015 (local address 127.0.0.1:38760)
0805 23:12:59.379621 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0805 23:12:59.379690 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.252:38015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260805 23:12:59.380596 7324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:12:59.380319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.252:38015 (local address 127.0.0.1:38764)
0805 23:12:59.380432 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0805 23:12:59.380491 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.252:38015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260805 23:12:59.380796 7320 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.92.252:38015: Network error: Client connection negotiation failed: client connection to 127.25.92.252:38015: connect: Connection refused (error 111)
I20260805 23:12:59.400655 7403 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.92.252" port: 38015 } attempt: 1
I20260805 23:12:59.401644 7405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:12:59.401275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.252:38015 (local address 127.0.0.1:38776)
0805 23:12:59.401426 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0805 23:12:59.401543 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.252:38015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260805 23:12:59.402519 7405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:12:59.402292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.252:38015 (local address 127.0.0.1:38782)
0805 23:12:59.402413 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0805 23:12:59.402466 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.252:38015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260805 23:12:59.402750 7403 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.92.252:38015: Network error: Client connection negotiation failed: client connection to 127.25.92.252:38015: connect: Connection refused (error 111)
I20260805 23:12:59.455994 7320 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.92.252" port: 38015 } attempt: 2
I20260805 23:12:59.456861 7324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:12:59.456543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.252:38015 (local address 127.0.0.1:38790)
0805 23:12:59.456713 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0805 23:12:59.456781 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.252:38015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260805 23:12:59.457726 7324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:12:59.457436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.252:38015 (local address 127.0.0.1:38792)
0805 23:12:59.457594 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0805 23:12:59.457643 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.252:38015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
W20260805 23:12:59.457895 7320 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.92.252:38015: Network error: Client connection negotiation failed: client connection to 127.25.92.252:38015: connect: Connection refused (error 111)
I20260805 23:12:59.470963 7403 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.92.252" port: 38015 } attempt: 2
I20260805 23:12:59.471901 7405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:12:59.471584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.252:38015 (local address 127.0.0.1:38798)
0805 23:12:59.471759 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0805 23:12:59.471822 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.252:38015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260805 23:12:59.472801 7405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:12:59.472553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.252:38015 (local address 127.0.0.1:38806)
0805 23:12:59.472691 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0805 23:12:59.472743 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.252:38015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260805 23:12:59.473124 7403 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.92.252:38015: Network error: Client connection negotiation failed: client connection to 127.25.92.252:38015: connect: Connection refused (error 111)
W20260805 23:12:59.478696 7411 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:12:59.478971 7411 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:12:59.479014 7411 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:12:59.483330 7411 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:12:59.483388 7411 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:12:59.483409 7411 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:12:59.483449 7411 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:12:59.483485 7411 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:12:59.489065 7411 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:33525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971406251336-25971-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971406251336-25971-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.25.92.254:46837,127.25.92.253:37205,127.25.92.252:38015
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.252.keytab
--principal=kudu/127.25.92.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.252:38015
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971406251336-25971-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.25.92.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7411
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971406251336-25971-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:12:59.490242 7411 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:12:59.491387 7411 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:12:59.500519 7421 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:12:59.500573 7417 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:12:59.500540 7419 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:12:59.500764 7411 server_base.cc:1034] running on GCE node
I20260805 23:12:59.501616 7411 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:12:59.502820 7411 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:12:59.504045 7411 hybrid_clock.cc:648] HybridClock initialized: now 1785971579504023 us; error 34 us; skew 500 ppm
Aug 05 23:12:59 dist-test-slave-ss4n krb5kdc[7232](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971579, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:12:59.507360 7411 init.cc:377] Logged in from keytab as kudu/127.25.92.252@KRBTEST.COM (short username kudu)
I20260805 23:12:59.508490 7411 webserver.cc:460] Webserver started at http://127.25.92.252:34863/ using document root <none> and password file <none>
I20260805 23:12:59.509114 7411 fs_manager.cc:359] Metadata directory not provided
I20260805 23:12:59.509218 7411 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:12:59.509518 7411 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:12:59.511826 7411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971406251336-25971-0/minicluster-data/master-2/data/instance:
uuid: "b9fe34eec3c14c2f892d79041c517fa9"
format_stamp: "Formatted at 2026-08-05 23:12:59 on dist-test-slave-ss4n"
server_key: "6304d4a4bee9e1892e01acfc8b0d85a5"
server_key_iv: "3c003dbcc506f2eb234e1c280152e456"
server_key_version: "encryptionkey@0"
I20260805 23:12:59.512465 7411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971406251336-25971-0/minicluster-data/master-2/wal/instance:
uuid: "b9fe34eec3c14c2f892d79041c517fa9"
format_stamp: "Formatted at 2026-08-05 23:12:59 on dist-test-slave-ss4n"
server_key: "6304d4a4bee9e1892e01acfc8b0d85a5"
server_key_iv: "3c003dbcc506f2eb234e1c280152e456"
server_key_version: "encryptionkey@0"
I20260805 23:12:59.516875 7411 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260805 23:12:59.520027 7429 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:12:59.521461 7411 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260805 23:12:59.521760 7411 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971406251336-25971-0/minicluster-data/master-2/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971406251336-25971-0/minicluster-data/master-2/wal
uuid: "b9fe34eec3c14c2f892d79041c517fa9"
format_stamp: "Formatted at 2026-08-05 23:12:59 on dist-test-slave-ss4n"
server_key: "6304d4a4bee9e1892e01acfc8b0d85a5"
server_key_iv: "3c003dbcc506f2eb234e1c280152e456"
server_key_version: "encryptionkey@0"
I20260805 23:12:59.521909 7411 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971406251336-25971-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971406251336-25971-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971406251336-25971-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:12:59.537405 7411 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:12:59.541416 7411 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:12:59.541745 7411 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:12:59.550642 7411 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.252:38015
I20260805 23:12:59.550714 7481 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.252:38015 every 8 connection(s)
I20260805 23:12:59.551887 7411 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971406251336-25971-0/minicluster-data/master-2/data/info.pb
I20260805 23:12:59.555137 7482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:12:59.557081 7320 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.92.252" port: 38015 } attempt: 3
I20260805 23:12:59.559861 7482 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.92.254" port: 46837 } has no permanent_uuid. Determining permanent_uuid...
I20260805 23:12:59.562311 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 7411
I20260805 23:12:59.562476 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971406251336-25971-0/minicluster-data/master-2/wal/instance
I20260805 23:12:59.562803 25971 external_mini_cluster.cc:1428] Setting key 492efe8e94c3cba3042b86d6a127af8f
I20260805 23:12:59.564338 7403 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.92.252" port: 38015 } attempt: 3
Aug 05 23:12:59 dist-test-slave-ss4n krb5kdc[7232](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971579, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.254@KRBTEST.COM for kudu/127.25.92.252@KRBTEST.COM
Aug 05 23:12:59 dist-test-slave-ss4n krb5kdc[7232](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971579, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.253@KRBTEST.COM for kudu/127.25.92.252@KRBTEST.COM
Aug 05 23:12:59 dist-test-slave-ss4n krb5kdc[7232](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971578, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.92.252@KRBTEST.COM
Aug 05 23:12:59 dist-test-slave-ss4n krb5kdc[7232](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971579, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.252@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM
I20260805 23:12:59.573336 7324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:12:59.557728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.252:38015 (local address 127.0.0.1:38822)
0805 23:12:59.557910 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0805 23:12:59.557931 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0805 23:12:59.558036 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:12:59.560529 (+ 2493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:12:59.560534 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:12:59.560754 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:12:59.560881 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:12:59.560890 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:12:59.562883 (+ 1993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:12:59.562887 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:12:59.563784 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:12:59.563799 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:12:59.565067 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:12:59.565074 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:12:59.565168 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:12:59.566200 (+ 1032us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:12:59.566219 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:12:59.567814 (+ 1595us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:12:59.570911 (+ 3097us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:12:59.570914 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:12:59.570916 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:12:59.571143 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:12:59.571594 (+ 451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:12:59.571598 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:12:59.571603 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:12:59.571708 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:12:59.572625 (+ 917us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:12:59.572629 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:12:59.572741 (+ 112us) client_negotiation.cc:769] Sending connection context
0805 23:12:59.572871 (+ 130us) client_negotiation.cc:240] Negotiation successful
0805 23:12:59.573026 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":71,"mutex_wait_us":36}
I20260805 23:12:59.574678 7484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:12:59.558556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38822 (local address 127.25.92.252:38015)
0805 23:12:59.559093 (+ 537us) server_negotiation.cc:206] Beginning negotiation
0805 23:12:59.559110 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0805 23:12:59.559147 (+ 37us) server_negotiation.cc:407] Connection header received
0805 23:12:59.559811 (+ 664us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:12:59.559829 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:12:59.560168 (+ 339us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:12:59.560531 (+ 363us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:12:59.561087 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:12:59.562713 (+ 1626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:12:59.563945 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:12:59.564714 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:12:59.564870 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:12:59.567979 (+ 3109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:12:59.567989 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:12:59.568004 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:12:59.568038 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:12:59.570777 (+ 2739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:12:59.571277 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:12:59.571287 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:12:59.571307 (+ 20us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:12:59.571465 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:12:59.571832 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:12:59.571836 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:12:59.571838 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:12:59.572470 (+ 632us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:12:59.572644 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0805 23:12:59.573050 (+ 406us) server_negotiation.cc:299] Negotiation successful
0805 23:12:59.573559 (+ 509us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":156,"threads_started":1}
I20260805 23:12:59.577210 7325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:12:59.561998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55750 (local address 127.25.92.254:46837)
0805 23:12:59.562194 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0805 23:12:59.562200 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:12:59.562626 (+ 426us) server_negotiation.cc:407] Connection header received
0805 23:12:59.562847 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:12:59.562850 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:12:59.562907 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:12:59.562961 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:12:59.563997 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:12:59.564809 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:12:59.566478 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:12:59.566792 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:12:59.566871 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:12:59.571874 (+ 5003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:12:59.571880 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:12:59.571886 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:12:59.571910 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:12:59.574969 (+ 3059us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:12:59.575517 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:12:59.575521 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:12:59.575523 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:12:59.575612 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:12:59.575940 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:12:59.575945 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:12:59.575950 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:12:59.576309 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:12:59.576389 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0805 23:12:59.576864 (+ 475us) server_negotiation.cc:299] Negotiation successful
0805 23:12:59.577010 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":61,"server-negotiator.queue_time_us":54}
I20260805 23:12:59.577358 7486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:12:59.563465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38830 (local address 127.25.92.252:38015)
0805 23:12:59.563901 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0805 23:12:59.563912 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0805 23:12:59.563955 (+ 43us) server_negotiation.cc:407] Connection header received
0805 23:12:59.564116 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:12:59.564119 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:12:59.564161 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:12:59.564246 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:12:59.565153 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:12:59.566381 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:12:59.567767 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:12:59.568113 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:12:59.568197 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:12:59.572295 (+ 4098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:12:59.572304 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:12:59.572307 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:12:59.572327 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:12:59.574850 (+ 2523us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:12:59.575424 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:12:59.575427 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:12:59.575429 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:12:59.575521 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:12:59.575872 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:12:59.575875 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:12:59.575877 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:12:59.576103 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:12:59.576275 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0805 23:12:59.576982 (+ 707us) server_negotiation.cc:299] Negotiation successful
0805 23:12:59.577107 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":103,"threads_started":1}
I20260805 23:12:59.578178 7405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:12:59.564823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.252:38015 (local address 127.0.0.1:38840)
0805 23:12:59.565000 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0805 23:12:59.565015 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0805 23:12:59.565098 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:12:59.565723 (+ 625us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:12:59.565729 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:12:59.565907 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:12:59.565995 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:12:59.566002 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:12:59.567071 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:12:59.567075 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:12:59.567888 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:12:59.567900 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:12:59.568491 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:12:59.568495 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:12:59.568608 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:12:59.569207 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:12:59.569224 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:12:59.571578 (+ 2354us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:12:59.576424 (+ 4846us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:12:59.576429 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:12:59.576431 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:12:59.576734 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:12:59.577061 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:12:59.577064 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:12:59.577067 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:12:59.577155 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:12:59.577679 (+ 524us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:12:59.577682 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:12:59.577785 (+ 103us) client_negotiation.cc:769] Sending connection context
0805 23:12:59.577867 (+ 82us) client_negotiation.cc:240] Negotiation successful
0805 23:12:59.578016 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":79}
I20260805 23:12:59.578485 7485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:12:59.561970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:46837 (local address 127.0.0.1:55750)
0805 23:12:59.562482 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0805 23:12:59.562545 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0805 23:12:59.562727 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:12:59.563106 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:12:59.563127 (+ 21us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:12:59.563652 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:12:59.563840 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:12:59.563849 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:12:59.565258 (+ 1409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:12:59.565260 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:12:59.566327 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:12:59.566340 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:12:59.567064 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:12:59.567068 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:12:59.567190 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:12:59.568070 (+ 880us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:12:59.568089 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:12:59.571715 (+ 3626us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:12:59.575087 (+ 3372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:12:59.575099 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:12:59.575104 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:12:59.575430 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:12:59.575717 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:12:59.575720 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:12:59.575722 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:12:59.575819 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:12:59.576474 (+ 655us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:12:59.576483 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:12:59.576610 (+ 127us) client_negotiation.cc:769] Sending connection context
0805 23:12:59.577878 (+ 1268us) client_negotiation.cc:240] Negotiation successful
0805 23:12:59.578064 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"spinlock_wait_cycles":3456,"thread_start_us":210,"threads_started":1}
I20260805 23:12:59.578749 7488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:12:59.565128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38840 (local address 127.25.92.252:38015)
0805 23:12:59.565417 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0805 23:12:59.565425 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0805 23:12:59.565438 (+ 13us) server_negotiation.cc:407] Connection header received
0805 23:12:59.565539 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:12:59.565542 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:12:59.565584 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:12:59.565688 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:12:59.566119 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:12:59.566918 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:12:59.568019 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:12:59.568353 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:12:59.568485 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:12:59.571731 (+ 3246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:12:59.571735 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:12:59.571737 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:12:59.571763 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:12:59.576257 (+ 4494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:12:59.576862 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:12:59.576865 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:12:59.576866 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:12:59.576946 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:12:59.577277 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:12:59.577283 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:12:59.577285 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:12:59.577547 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:12:59.578379 (+ 832us) server_negotiation.cc:1035] Waiting for connection context
0805 23:12:59.578455 (+ 76us) server_negotiation.cc:299] Negotiation successful
0805 23:12:59.578577 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":220,"thread_start_us":100,"threads_started":1}
I20260805 23:12:59.580041 7320 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b1e8f104b67405f97ccc1d3c7335629: Bootstrap starting.
I20260805 23:12:59.582697 7482 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.92.253" port: 37205 } has no permanent_uuid. Determining permanent_uuid...
I20260805 23:12:59.582948 7320 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0b1e8f104b67405f97ccc1d3c7335629: Neither blocks nor log segments found. Creating new log.
I20260805 23:12:59.583886 7320 log.cc:824] T 00000000000000000000000000000000 P 0b1e8f104b67405f97ccc1d3c7335629: Log is configured to *not* fsync() on all Append() calls
Aug 05 23:12:59 dist-test-slave-ss4n krb5kdc[7232](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971578, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM
I20260805 23:12:59.586309 7320 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b1e8f104b67405f97ccc1d3c7335629: No bootstrap required, opened a new log
I20260805 23:12:59.586453 7403 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 802657d49388472a8de98cd7b46e2361: Bootstrap starting.
I20260805 23:12:59.589923 7325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:12:59.578651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55756 (local address 127.25.92.254:46837)
0805 23:12:59.578805 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0805 23:12:59.578809 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:12:59.579049 (+ 240us) server_negotiation.cc:407] Connection header received
0805 23:12:59.579086 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:12:59.579088 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:12:59.579170 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:12:59.579254 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:12:59.580010 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:12:59.580602 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:12:59.581693 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:12:59.582034 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:12:59.582142 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:12:59.585446 (+ 3304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:12:59.585450 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:12:59.585453 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:12:59.585478 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:12:59.587445 (+ 1967us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:12:59.588010 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:12:59.588016 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:12:59.588019 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:12:59.588115 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:12:59.588437 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:12:59.588440 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:12:59.588442 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:12:59.588666 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:12:59.588831 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0805 23:12:59.589623 (+ 792us) server_negotiation.cc:299] Negotiation successful
0805 23:12:59.589751 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260805 23:12:59.590142 7403 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 802657d49388472a8de98cd7b46e2361: Neither blocks nor log segments found. Creating new log.
I20260805 23:12:59.590715 7320 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0b1e8f104b67405f97ccc1d3c7335629 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b1e8f104b67405f97ccc1d3c7335629" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 46837 } } peers { permanent_uuid: "802657d49388472a8de98cd7b46e2361" member_type: VOTER last_known_addr { host: "127.25.92.253" port: 37205 } } peers { permanent_uuid: "b9fe34eec3c14c2f892d79041c517fa9" member_type: VOTER last_known_addr { host: "127.25.92.252" port: 38015 } }
I20260805 23:12:59.590966 7320 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0b1e8f104b67405f97ccc1d3c7335629 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:12:59.591032 7320 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0b1e8f104b67405f97ccc1d3c7335629 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b1e8f104b67405f97ccc1d3c7335629, State: Initialized, Role: FOLLOWER
I20260805 23:12:59.591084 7403 log.cc:824] T 00000000000000000000000000000000 P 802657d49388472a8de98cd7b46e2361: Log is configured to *not* fsync() on all Append() calls
Aug 05 23:12:59 dist-test-slave-ss4n krb5kdc[7232](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971579, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.252@KRBTEST.COM for kudu/127.25.92.253@KRBTEST.COM
I20260805 23:12:59.591567 7320 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0b1e8f104b67405f97ccc1d3c7335629 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b1e8f104b67405f97ccc1d3c7335629" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 46837 } } peers { permanent_uuid: "802657d49388472a8de98cd7b46e2361" member_type: VOTER last_known_addr { host: "127.25.92.253" port: 37205 } } peers { permanent_uuid: "b9fe34eec3c14c2f892d79041c517fa9" member_type: VOTER last_known_addr { host: "127.25.92.252" port: 38015 } }
I20260805 23:12:59.593151 7320 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0b1e8f104b67405f97ccc1d3c7335629 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:12:59.593740 7491 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b1e8f104b67405f97ccc1d3c7335629 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b1e8f104b67405f97ccc1d3c7335629" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 46837 } } peers { permanent_uuid: "802657d49388472a8de98cd7b46e2361" member_type: VOTER last_known_addr { host: "127.25.92.253" port: 37205 } } peers { permanent_uuid: "b9fe34eec3c14c2f892d79041c517fa9" member_type: VOTER last_known_addr { host: "127.25.92.252" port: 38015 } } }
I20260805 23:12:59.593997 7491 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b1e8f104b67405f97ccc1d3c7335629 [sys.catalog]: This master's current role is: FOLLOWER
I20260805 23:12:59.596817 7403 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 802657d49388472a8de98cd7b46e2361: No bootstrap required, opened a new log
Aug 05 23:12:59 dist-test-slave-ss4n krb5kdc[7232](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971578, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.92.253@KRBTEST.COM
W20260805 23:12:59.599620 7502 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0b1e8f104b67405f97ccc1d3c7335629: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260805 23:12:59.599751 7502 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260805 23:12:59.601454 7403 raft_consensus.cc:348] T 00000000000000000000000000000000 P 802657d49388472a8de98cd7b46e2361 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b1e8f104b67405f97ccc1d3c7335629" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 46837 } } peers { permanent_uuid: "802657d49388472a8de98cd7b46e2361" member_type: VOTER last_known_addr { host: "127.25.92.253" port: 37205 } } peers { permanent_uuid: "b9fe34eec3c14c2f892d79041c517fa9" member_type: VOTER last_known_addr { host: "127.25.92.252" port: 38015 } }
I20260805 23:12:59.601723 7403 raft_consensus.cc:374] T 00000000000000000000000000000000 P 802657d49388472a8de98cd7b46e2361 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:12:59.601783 7403 raft_consensus.cc:729] T 00000000000000000000000000000000 P 802657d49388472a8de98cd7b46e2361 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 802657d49388472a8de98cd7b46e2361, State: Initialized, Role: FOLLOWER
I20260805 23:12:59.598517 7246 master_runner.cc:186] Error getting master registration for 127.25.92.253:37205: OK, instance_id { permanent_uuid: "802657d49388472a8de98cd7b46e2361" instance_seqno: 1785971579307088 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260805 23:12:59.602138 7407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:12:59.591347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46002 (local address 127.25.92.253:37205)
0805 23:12:59.591462 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0805 23:12:59.591466 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:12:59.591749 (+ 283us) server_negotiation.cc:407] Connection header received
0805 23:12:59.591789 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:12:59.591792 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:12:59.591839 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:12:59.591917 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:12:59.592736 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:12:59.593562 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:12:59.594679 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:12:59.595004 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:12:59.595097 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:12:59.598129 (+ 3032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:12:59.598133 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:12:59.598136 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:12:59.598160 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:12:59.599850 (+ 1690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:12:59.600346 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:12:59.600349 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:12:59.600351 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:12:59.600435 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:12:59.600744 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:12:59.600747 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:12:59.600749 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:12:59.600973 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:12:59.601156 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0805 23:12:59.601836 (+ 680us) server_negotiation.cc:299] Negotiation successful
0805 23:12:59.601957 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260805 23:12:59.602222 7246 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260805 23:12:59.603011 7409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:12:59.585622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45992 (local address 127.25.92.253:37205)
0805 23:12:59.585865 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0805 23:12:59.585870 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:12:59.585885 (+ 15us) server_negotiation.cc:407] Connection header received
0805 23:12:59.585959 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:12:59.585964 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:12:59.586018 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:12:59.586103 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:12:59.586612 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:12:59.587368 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:12:59.588501 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:12:59.588855 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:12:59.588949 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:12:59.591972 (+ 3023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:12:59.591975 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:12:59.591978 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:12:59.592003 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:12:59.593779 (+ 1776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:12:59.594284 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:12:59.594288 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:12:59.594290 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:12:59.594370 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:12:59.602105 (+ 7735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:12:59.602109 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:12:59.602111 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:12:59.602349 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:12:59.602410 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0805 23:12:59.602764 (+ 354us) server_negotiation.cc:299] Negotiation successful
0805 23:12:59.602873 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145}
I20260805 23:12:59.603019 7485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:12:59.583247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.253:37205 (local address 127.0.0.1:45992)
0805 23:12:59.585773 (+ 2526us) negotiation.cc:107] Waiting for socket to connect
0805 23:12:59.585791 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0805 23:12:59.585872 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:12:59.586147 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:12:59.586151 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:12:59.586337 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:12:59.586470 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:12:59.586478 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:12:59.587522 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:12:59.587525 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:12:59.588355 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:12:59.588366 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:12:59.589144 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:12:59.589147 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:12:59.589266 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:12:59.589796 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:12:59.589812 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:12:59.591775 (+ 1963us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:12:59.593895 (+ 2120us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:12:59.593898 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:12:59.593901 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:12:59.594168 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:12:59.594475 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:12:59.594478 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:12:59.594480 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:12:59.594567 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:12:59.602494 (+ 7927us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:12:59.602497 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:12:59.602588 (+ 91us) client_negotiation.cc:769] Sending connection context
0805 23:12:59.602697 (+ 109us) client_negotiation.cc:240] Negotiation successful
0805 23:12:59.602865 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2441}
I20260805 23:12:59.603574 7403 consensus_queue.cc:260] T 00000000000000000000000000000000 P 802657d49388472a8de98cd7b46e2361 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b1e8f104b67405f97ccc1d3c7335629" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 46837 } } peers { permanent_uuid: "802657d49388472a8de98cd7b46e2361" member_type: VOTER last_known_addr { host: "127.25.92.253" port: 37205 } } peers { permanent_uuid: "b9fe34eec3c14c2f892d79041c517fa9" member_type: VOTER last_known_addr { host: "127.25.92.252" port: 38015 } }
I20260805 23:12:59.603736 7482 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.92.252" port: 38015 } has no permanent_uuid. Determining permanent_uuid...
I20260805 23:12:59.605363 25971 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260805 23:12:59.605468 7403 sys_catalog.cc:564] T 00000000000000000000000000000000 P 802657d49388472a8de98cd7b46e2361 [sys.catalog]: configured and running, proceeding with master startup.
Aug 05 23:12:59 dist-test-slave-ss4n krb5kdc[7232](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971579, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.252@KRBTEST.COM for kudu/127.25.92.252@KRBTEST.COM
W20260805 23:12:59.611950 7520 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 802657d49388472a8de98cd7b46e2361: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260805 23:12:59.612042 7520 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260805 23:12:59.612366 7504 sys_catalog.cc:455] T 00000000000000000000000000000000 P 802657d49388472a8de98cd7b46e2361 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b1e8f104b67405f97ccc1d3c7335629" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 46837 } } peers { permanent_uuid: "802657d49388472a8de98cd7b46e2361" member_type: VOTER last_known_addr { host: "127.25.92.253" port: 37205 } } peers { permanent_uuid: "b9fe34eec3c14c2f892d79041c517fa9" member_type: VOTER last_known_addr { host: "127.25.92.252" port: 38015 } } }
I20260805 23:12:59.612497 7504 sys_catalog.cc:458] T 00000000000000000000000000000000 P 802657d49388472a8de98cd7b46e2361 [sys.catalog]: This master's current role is: FOLLOWER
I20260805 23:12:59.613780 7485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:12:59.604186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.252:38015 (local address 127.0.0.1:38844)
0805 23:12:59.604288 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0805 23:12:59.604303 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0805 23:12:59.604475 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:12:59.604755 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:12:59.604757 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:12:59.604928 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:12:59.605013 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:12:59.605020 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:12:59.606028 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:12:59.606031 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:12:59.606893 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:12:59.606908 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:12:59.607694 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:12:59.607697 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:12:59.607788 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:12:59.608535 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:12:59.608549 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:12:59.610132 (+ 1583us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:12:59.612121 (+ 1989us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:12:59.612124 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:12:59.612127 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:12:59.612388 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:12:59.612728 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:12:59.612731 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:12:59.612734 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:12:59.612818 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:12:59.613234 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:12:59.613236 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:12:59.613317 (+ 81us) client_negotiation.cc:769] Sending connection context
0805 23:12:59.613413 (+ 96us) client_negotiation.cc:240] Negotiation successful
0805 23:12:59.613593 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260805 23:12:59.614301 7486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:12:59.604361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38844 (local address 127.25.92.252:38015)
0805 23:12:59.604451 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0805 23:12:59.604455 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:12:59.604467 (+ 12us) server_negotiation.cc:407] Connection header received
0805 23:12:59.604548 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:12:59.604552 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:12:59.604597 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:12:59.604661 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:12:59.605135 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:12:59.605909 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:12:59.607033 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:12:59.607410 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:12:59.607504 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:12:59.610268 (+ 2764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:12:59.610272 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:12:59.610274 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:12:59.610295 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:12:59.612003 (+ 1708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:12:59.612514 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:12:59.612518 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:12:59.612520 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:12:59.612626 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:12:59.612899 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:12:59.612902 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:12:59.612904 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:12:59.613111 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:12:59.613201 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0805 23:12:59.614043 (+ 842us) server_negotiation.cc:299] Negotiation successful
0805 23:12:59.614161 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260805 23:12:59.621752 7482 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9fe34eec3c14c2f892d79041c517fa9: Bootstrap starting.
I20260805 23:12:59.622128 7325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:12:59.608845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55770 (local address 127.25.92.254:46837)
0805 23:12:59.609102 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0805 23:12:59.609106 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:12:59.611684 (+ 2578us) server_negotiation.cc:407] Connection header received
0805 23:12:59.612654 (+ 970us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:12:59.612657 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:12:59.612712 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:12:59.612764 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:12:59.613827 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:12:59.614567 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:12:59.615718 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:12:59.616043 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:12:59.616118 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:12:59.618005 (+ 1887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:12:59.618010 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:12:59.618013 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:12:59.618042 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:12:59.619943 (+ 1901us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:12:59.620516 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:12:59.620521 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:12:59.620525 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:12:59.620640 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:12:59.620966 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:12:59.620969 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:12:59.620971 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:12:59.621227 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:12:59.621366 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0805 23:12:59.621806 (+ 440us) server_negotiation.cc:299] Negotiation successful
0805 23:12:59.621963 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185}
I20260805 23:12:59.623358 7409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:12:59.607782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46008 (local address 127.25.92.253:37205)
0805 23:12:59.607898 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0805 23:12:59.607902 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:12:59.611228 (+ 3326us) server_negotiation.cc:407] Connection header received
0805 23:12:59.612288 (+ 1060us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:12:59.612292 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:12:59.612342 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:12:59.612397 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:12:59.615316 (+ 2919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:12:59.616071 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:12:59.617242 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:12:59.617614 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:12:59.617694 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:12:59.619489 (+ 1795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:12:59.619493 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:12:59.619496 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:12:59.619525 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:12:59.621281 (+ 1756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:12:59.621884 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:12:59.621888 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:12:59.621890 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:12:59.621981 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:12:59.622332 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:12:59.622336 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:12:59.622338 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:12:59.622560 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:12:59.622711 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0805 23:12:59.623071 (+ 360us) server_negotiation.cc:299] Negotiation successful
0805 23:12:59.623203 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260805 23:12:59.616880 7328 master_runner.cc:186] Error getting master registration for 127.25.92.252:38015: OK, instance_id { permanent_uuid: "b9fe34eec3c14c2f892d79041c517fa9" instance_seqno: 1785971579547824 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260805 23:12:59.627378 7328 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260805 23:12:59.629261 7482 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b9fe34eec3c14c2f892d79041c517fa9: Neither blocks nor log segments found. Creating new log.
I20260805 23:12:59.630337 7482 log.cc:824] T 00000000000000000000000000000000 P b9fe34eec3c14c2f892d79041c517fa9: Log is configured to *not* fsync() on all Append() calls
I20260805 23:12:59.633710 7488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:12:59.611022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38860 (local address 127.25.92.252:38015)
0805 23:12:59.611143 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0805 23:12:59.611148 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:12:59.624026 (+ 12878us) server_negotiation.cc:407] Connection header received
0805 23:12:59.624212 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:12:59.624215 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:12:59.624267 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:12:59.624319 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:12:59.625229 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:12:59.626097 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:12:59.627417 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:12:59.627771 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:12:59.627852 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:12:59.629793 (+ 1941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:12:59.629797 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:12:59.629800 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:12:59.629829 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:12:59.631630 (+ 1801us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:12:59.632158 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:12:59.632162 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:12:59.632164 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:12:59.632258 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:12:59.632586 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:12:59.632589 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:12:59.632591 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:12:59.632838 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:12:59.632961 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0805 23:12:59.633361 (+ 400us) server_negotiation.cc:299] Negotiation successful
0805 23:12:59.633533 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260805 23:12:59.635993 7482 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9fe34eec3c14c2f892d79041c517fa9: No bootstrap required, opened a new log
I20260805 23:12:59.641590 7482 raft_consensus.cc:348] T 00000000000000000000000000000000 P b9fe34eec3c14c2f892d79041c517fa9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b1e8f104b67405f97ccc1d3c7335629" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 46837 } } peers { permanent_uuid: "802657d49388472a8de98cd7b46e2361" member_type: VOTER last_known_addr { host: "127.25.92.253" port: 37205 } } peers { permanent_uuid: "b9fe34eec3c14c2f892d79041c517fa9" member_type: VOTER last_known_addr { host: "127.25.92.252" port: 38015 } }
I20260805 23:12:59.641896 7482 raft_consensus.cc:374] T 00000000000000000000000000000000 P b9fe34eec3c14c2f892d79041c517fa9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:12:59.641947 7482 raft_consensus.cc:729] T 00000000000000000000000000000000 P b9fe34eec3c14c2f892d79041c517fa9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9fe34eec3c14c2f892d79041c517fa9, State: Initialized, Role: FOLLOWER
I20260805 23:12:59.642518 7482 consensus_queue.cc:260] T 00000000000000000000000000000000 P b9fe34eec3c14c2f892d79041c517fa9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b1e8f104b67405f97ccc1d3c7335629" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 46837 } } peers { permanent_uuid: "802657d49388472a8de98cd7b46e2361" member_type: VOTER last_known_addr { host: "127.25.92.253" port: 37205 } } peers { permanent_uuid: "b9fe34eec3c14c2f892d79041c517fa9" member_type: VOTER last_known_addr { host: "127.25.92.252" port: 38015 } }
I20260805 23:12:59.644129 7482 sys_catalog.cc:564] T 00000000000000000000000000000000 P b9fe34eec3c14c2f892d79041c517fa9 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:12:59.648303 7526 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9fe34eec3c14c2f892d79041c517fa9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b1e8f104b67405f97ccc1d3c7335629" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 46837 } } peers { permanent_uuid: "802657d49388472a8de98cd7b46e2361" member_type: VOTER last_known_addr { host: "127.25.92.253" port: 37205 } } peers { permanent_uuid: "b9fe34eec3c14c2f892d79041c517fa9" member_type: VOTER last_known_addr { host: "127.25.92.252" port: 38015 } } }
I20260805 23:12:59.648492 7526 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9fe34eec3c14c2f892d79041c517fa9 [sys.catalog]: This master's current role is: FOLLOWER
I20260805 23:12:59.653982 7411 master_runner.cc:263] No leader master found from master b9fe34eec3c14c2f892d79041c517fa9
I20260805 23:12:59.654086 7411 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260805 23:12:59.654327 7537 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b9fe34eec3c14c2f892d79041c517fa9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260805 23:12:59.654484 7537 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260805 23:12:59.793346 7435 tablet.cc:2369] T 00000000000000000000000000000000 P b9fe34eec3c14c2f892d79041c517fa9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260805 23:12:59.803102 7352 tablet.cc:2369] T 00000000000000000000000000000000 P 802657d49388472a8de98cd7b46e2361: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260805 23:12:59.822629 7504 raft_consensus.cc:482] T 00000000000000000000000000000000 P 802657d49388472a8de98cd7b46e2361 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 23:12:59.822911 7504 raft_consensus.cc:504] T 00000000000000000000000000000000 P 802657d49388472a8de98cd7b46e2361 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b1e8f104b67405f97ccc1d3c7335629" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 46837 } } peers { permanent_uuid: "802657d49388472a8de98cd7b46e2361" member_type: VOTER last_known_addr { host: "127.25.92.253" port: 37205 } } peers { permanent_uuid: "b9fe34eec3c14c2f892d79041c517fa9" member_type: VOTER last_known_addr { host: "127.25.92.252" port: 38015 } }
I20260805 23:12:59.824096 7504 leader_election.cc:288] T 00000000000000000000000000000000 P 802657d49388472a8de98cd7b46e2361 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0b1e8f104b67405f97ccc1d3c7335629 (127.25.92.254:46837), b9fe34eec3c14c2f892d79041c517fa9 (127.25.92.252:38015)
I20260805 23:12:59.824774 7456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "802657d49388472a8de98cd7b46e2361" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9fe34eec3c14c2f892d79041c517fa9" is_pre_election: true
I20260805 23:12:59.824769 7291 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "802657d49388472a8de98cd7b46e2361" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b1e8f104b67405f97ccc1d3c7335629" is_pre_election: true
I20260805 23:12:59.825137 7456 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b9fe34eec3c14c2f892d79041c517fa9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 802657d49388472a8de98cd7b46e2361 in term 0.
I20260805 23:12:59.825138 7291 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0b1e8f104b67405f97ccc1d3c7335629 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 802657d49388472a8de98cd7b46e2361 in term 0.
I20260805 23:12:59.825806 7346 leader_election.cc:302] T 00000000000000000000000000000000 P 802657d49388472a8de98cd7b46e2361 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0b1e8f104b67405f97ccc1d3c7335629, 802657d49388472a8de98cd7b46e2361, b9fe34eec3c14c2f892d79041c517fa9; no voters:
I20260805 23:12:59.826243 7504 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 802657d49388472a8de98cd7b46e2361 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260805 23:12:59.826339 7504 raft_consensus.cc:482] T 00000000000000000000000000000000 P 802657d49388472a8de98cd7b46e2361 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260805 23:12:59.826398 7504 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 802657d49388472a8de98cd7b46e2361 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:12:59.827827 7504 raft_consensus.cc:504] T 00000000000000000000000000000000 P 802657d49388472a8de98cd7b46e2361 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b1e8f104b67405f97ccc1d3c7335629" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 46837 } } peers { permanent_uuid: "802657d49388472a8de98cd7b46e2361" member_type: VOTER last_known_addr { host: "127.25.92.253" port: 37205 } } peers { permanent_uuid: "b9fe34eec3c14c2f892d79041c517fa9" member_type: VOTER last_known_addr { host: "127.25.92.252" port: 38015 } }
I20260805 23:12:59.828306 7504 leader_election.cc:288] T 00000000000000000000000000000000 P 802657d49388472a8de98cd7b46e2361 [CANDIDATE]: Term 1 election: Requested vote from peers 0b1e8f104b67405f97ccc1d3c7335629 (127.25.92.254:46837), b9fe34eec3c14c2f892d79041c517fa9 (127.25.92.252:38015)
I20260805 23:12:59.828800 7456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "802657d49388472a8de98cd7b46e2361" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9fe34eec3c14c2f892d79041c517fa9"
I20260805 23:12:59.828801 7291 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "802657d49388472a8de98cd7b46e2361" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b1e8f104b67405f97ccc1d3c7335629"
I20260805 23:12:59.828963 7456 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b9fe34eec3c14c2f892d79041c517fa9 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:12:59.828949 7291 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0b1e8f104b67405f97ccc1d3c7335629 [term 0 FOLLOWER]: Advancing to term 1
W20260805 23:12:59.829466 7268 tablet.cc:2369] T 00000000000000000000000000000000 P 0b1e8f104b67405f97ccc1d3c7335629: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260805 23:12:59.830438 7291 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0b1e8f104b67405f97ccc1d3c7335629 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 802657d49388472a8de98cd7b46e2361 in term 1.
I20260805 23:12:59.830438 7456 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b9fe34eec3c14c2f892d79041c517fa9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 802657d49388472a8de98cd7b46e2361 in term 1.
I20260805 23:12:59.830847 7346 leader_election.cc:302] T 00000000000000000000000000000000 P 802657d49388472a8de98cd7b46e2361 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 802657d49388472a8de98cd7b46e2361, b9fe34eec3c14c2f892d79041c517fa9; no voters:
I20260805 23:12:59.831080 7504 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 802657d49388472a8de98cd7b46e2361 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:12:59.831363 7504 raft_consensus.cc:686] T 00000000000000000000000000000000 P 802657d49388472a8de98cd7b46e2361 [term 1 LEADER]: Becoming Leader. State: Replica: 802657d49388472a8de98cd7b46e2361, State: Running, Role: LEADER
I20260805 23:12:59.831745 7504 consensus_queue.cc:237] T 00000000000000000000000000000000 P 802657d49388472a8de98cd7b46e2361 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b1e8f104b67405f97ccc1d3c7335629" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 46837 } } peers { permanent_uuid: "802657d49388472a8de98cd7b46e2361" member_type: VOTER last_known_addr { host: "127.25.92.253" port: 37205 } } peers { permanent_uuid: "b9fe34eec3c14c2f892d79041c517fa9" member_type: VOTER last_known_addr { host: "127.25.92.252" port: 38015 } }
I20260805 23:12:59.835201 7539 sys_catalog.cc:455] T 00000000000000000000000000000000 P 802657d49388472a8de98cd7b46e2361 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 802657d49388472a8de98cd7b46e2361. Latest consensus state: current_term: 1 leader_uuid: "802657d49388472a8de98cd7b46e2361" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b1e8f104b67405f97ccc1d3c7335629" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 46837 } } peers { permanent_uuid: "802657d49388472a8de98cd7b46e2361" member_type: VOTER last_known_addr { host: "127.25.92.253" port: 37205 } } peers { permanent_uuid: "b9fe34eec3c14c2f892d79041c517fa9" member_type: VOTER last_known_addr { host: "127.25.92.252" port: 38015 } } }
I20260805 23:12:59.835460 7539 sys_catalog.cc:458] T 00000000000000000000000000000000 P 802657d49388472a8de98cd7b46e2361 [sys.catalog]: This master's current role is: LEADER
I20260805 23:12:59.837139 7541 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:12:59.838094 7541 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:12:59.843480 7456 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b9fe34eec3c14c2f892d79041c517fa9 [term 1 FOLLOWER]: Refusing update from remote peer 802657d49388472a8de98cd7b46e2361: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:12:59.843529 7291 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0b1e8f104b67405f97ccc1d3c7335629 [term 1 FOLLOWER]: Refusing update from remote peer 802657d49388472a8de98cd7b46e2361: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:12:59.844322 7539 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 802657d49388472a8de98cd7b46e2361 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b9fe34eec3c14c2f892d79041c517fa9" member_type: VOTER last_known_addr { host: "127.25.92.252" port: 38015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:12:59.844585 7504 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 802657d49388472a8de98cd7b46e2361 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b1e8f104b67405f97ccc1d3c7335629" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 46837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:12:59.848919 7491 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b1e8f104b67405f97ccc1d3c7335629 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 802657d49388472a8de98cd7b46e2361. Latest consensus state: current_term: 1 leader_uuid: "802657d49388472a8de98cd7b46e2361" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b1e8f104b67405f97ccc1d3c7335629" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 46837 } } peers { permanent_uuid: "802657d49388472a8de98cd7b46e2361" member_type: VOTER last_known_addr { host: "127.25.92.253" port: 37205 } } peers { permanent_uuid: "b9fe34eec3c14c2f892d79041c517fa9" member_type: VOTER last_known_addr { host: "127.25.92.252" port: 38015 } } }
I20260805 23:12:59.849130 7491 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b1e8f104b67405f97ccc1d3c7335629 [sys.catalog]: This master's current role is: FOLLOWER
I20260805 23:12:59.849356 7526 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9fe34eec3c14c2f892d79041c517fa9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 802657d49388472a8de98cd7b46e2361. Latest consensus state: current_term: 1 leader_uuid: "802657d49388472a8de98cd7b46e2361" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b1e8f104b67405f97ccc1d3c7335629" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 46837 } } peers { permanent_uuid: "802657d49388472a8de98cd7b46e2361" member_type: VOTER last_known_addr { host: "127.25.92.253" port: 37205 } } peers { permanent_uuid: "b9fe34eec3c14c2f892d79041c517fa9" member_type: VOTER last_known_addr { host: "127.25.92.252" port: 38015 } } }
I20260805 23:12:59.849546 7526 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9fe34eec3c14c2f892d79041c517fa9 [sys.catalog]: This master's current role is: FOLLOWER
I20260805 23:12:59.852748 7548 sys_catalog.cc:455] T 00000000000000000000000000000000 P 802657d49388472a8de98cd7b46e2361 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "802657d49388472a8de98cd7b46e2361" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b1e8f104b67405f97ccc1d3c7335629" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 46837 } } peers { permanent_uuid: "802657d49388472a8de98cd7b46e2361" member_type: VOTER last_known_addr { host: "127.25.92.253" port: 37205 } } peers { permanent_uuid: "b9fe34eec3c14c2f892d79041c517fa9" member_type: VOTER last_known_addr { host: "127.25.92.252" port: 38015 } } }
I20260805 23:12:59.852897 7548 sys_catalog.cc:458] T 00000000000000000000000000000000 P 802657d49388472a8de98cd7b46e2361 [sys.catalog]: This master's current role is: LEADER
I20260805 23:12:59.853021 7548 sys_catalog.cc:455] T 00000000000000000000000000000000 P 802657d49388472a8de98cd7b46e2361 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "802657d49388472a8de98cd7b46e2361" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b1e8f104b67405f97ccc1d3c7335629" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 46837 } } peers { permanent_uuid: "802657d49388472a8de98cd7b46e2361" member_type: VOTER last_known_addr { host: "127.25.92.253" port: 37205 } } peers { permanent_uuid: "b9fe34eec3c14c2f892d79041c517fa9" member_type: VOTER last_known_addr { host: "127.25.92.252" port: 38015 } } }
I20260805 23:12:59.853087 7548 sys_catalog.cc:458] T 00000000000000000000000000000000 P 802657d49388472a8de98cd7b46e2361 [sys.catalog]: This master's current role is: LEADER
I20260805 23:12:59.853190 7544 mvcc.cc:205] Tried to move back new op lower bound from 7315339591030943744 to 7315339590999023616. Current Snapshot: MvccSnapshot[applied={T|T < 7315339591030943744}]
I20260805 23:12:59.853363 7526 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9fe34eec3c14c2f892d79041c517fa9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "802657d49388472a8de98cd7b46e2361" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b1e8f104b67405f97ccc1d3c7335629" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 46837 } } peers { permanent_uuid: "802657d49388472a8de98cd7b46e2361" member_type: VOTER last_known_addr { host: "127.25.92.253" port: 37205 } } peers { permanent_uuid: "b9fe34eec3c14c2f892d79041c517fa9" member_type: VOTER last_known_addr { host: "127.25.92.252" port: 38015 } } }
I20260805 23:12:59.853475 7526 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9fe34eec3c14c2f892d79041c517fa9 [sys.catalog]: This master's current role is: FOLLOWER
I20260805 23:12:59.853453 7545 mvcc.cc:205] Tried to move back new op lower bound from 7315339591030943744 to 7315339590999023616. Current Snapshot: MvccSnapshot[applied={T|T < 7315339591030943744}]
I20260805 23:12:59.853739 7491 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b1e8f104b67405f97ccc1d3c7335629 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "802657d49388472a8de98cd7b46e2361" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b1e8f104b67405f97ccc1d3c7335629" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 46837 } } peers { permanent_uuid: "802657d49388472a8de98cd7b46e2361" member_type: VOTER last_known_addr { host: "127.25.92.253" port: 37205 } } peers { permanent_uuid: "b9fe34eec3c14c2f892d79041c517fa9" member_type: VOTER last_known_addr { host: "127.25.92.252" port: 38015 } } }
I20260805 23:12:59.853874 7491 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b1e8f104b67405f97ccc1d3c7335629 [sys.catalog]: This master's current role is: FOLLOWER
I20260805 23:12:59.856372 7541 catalog_manager.cc:1347] Generated new cluster ID: c03a81621780433ab99779bf120d96e4
I20260805 23:12:59.856457 7541 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:12:59.883791 7541 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:12:59.885257 7541 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:12:59.893137 7541 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 802657d49388472a8de98cd7b46e2361: Generated new TSK 0
I20260805 23:12:59.894203 7541 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260805 23:13:00.601146 7502 catalog_manager.cc:1259] Loaded cluster ID: c03a81621780433ab99779bf120d96e4
I20260805 23:13:00.601269 7502 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 0b1e8f104b67405f97ccc1d3c7335629: loading cluster ID for follower catalog manager: success
I20260805 23:13:00.603003 7502 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 0b1e8f104b67405f97ccc1d3c7335629: acquiring CA information for follower catalog manager: success
I20260805 23:13:00.604195 7502 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 0b1e8f104b67405f97ccc1d3c7335629: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260805 23:13:00.656045 7537 catalog_manager.cc:1259] Loaded cluster ID: c03a81621780433ab99779bf120d96e4
I20260805 23:13:00.656150 7537 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b9fe34eec3c14c2f892d79041c517fa9: loading cluster ID for follower catalog manager: success
I20260805 23:13:00.657976 7537 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b9fe34eec3c14c2f892d79041c517fa9: acquiring CA information for follower catalog manager: success
I20260805 23:13:00.659197 7537 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b9fe34eec3c14c2f892d79041c517fa9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260805 23:13:00.963554 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 7246
I20260805 23:13:00.970762 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 7328
I20260805 23:13:00.977454 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 7411
2026-08-05T23:13:00Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3336 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785971406251336-25971-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 23:13:01 dist-test-slave-ss4n krb5kdc[7558](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 23:13:01 dist-test-slave-ss4n krb5kdc[7558](info): set up 2 sockets
Aug 05 23:13:01 dist-test-slave-ss4n krb5kdc[7558](info): commencing operation
krb5kdc: starting...
W20260805 23:13:02.068827 25971 mini_kdc.cc:121] Time spent starting KDC: real 1.068s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785971406251336-25971-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785971406251336-25971-0/krb5kdc/test-user.keytab.
Aug 05 23:13:02 dist-test-slave-ss4n krb5kdc[7558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971582, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T23:13:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:13:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.92.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.92.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.92.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab.
Entry for principal kudu/127.25.92.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab.
I20260805 23:13:02.217895 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785971406251336-25971-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785971406251336-25971-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785971406251336-25971-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.254:37317
--webserver_interface=127.25.92.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab
--principal=kudu/127.25.92.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:35355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.92.254:37317
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:13:02.335441 7572 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:13:02.335772 7572 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:13:02.335815 7572 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:13:02.341219 7572 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:13:02.341333 7572 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:13:02.341357 7572 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:13:02.341377 7572 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:13:02.341395 7572 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:13:02.348575 7572 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:35355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785971406251336-25971-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785971406251336-25971-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.92.254:37317
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab
--principal=kudu/127.25.92.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.254:37317
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.92.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7572
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785971406251336-25971-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:13:02.349965 7572 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:13:02.351195 7572 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:13:02.361690 7577 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:13:02.361749 7572 server_base.cc:1034] running on GCE node
W20260805 23:13:02.361691 7580 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:13:02.361691 7578 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:13:02.362804 7572 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:13:02.364127 7572 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:13:02.365546 7572 hybrid_clock.cc:648] HybridClock initialized: now 1785971582365362 us; error 65 us; skew 500 ppm
Aug 05 23:13:02 dist-test-slave-ss4n krb5kdc[7558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971582, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:13:02.369143 7572 init.cc:377] Logged in from keytab as kudu/127.25.92.254@KRBTEST.COM (short username kudu)
I20260805 23:13:02.370225 7572 webserver.cc:460] Webserver started at http://127.25.92.254:45901/ using document root <none> and password file <none>
I20260805 23:13:02.370895 7572 fs_manager.cc:359] Metadata directory not provided
I20260805 23:13:02.370970 7572 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:13:02.371155 7572 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:13:02.373358 7572 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785971406251336-25971-0/minicluster-data/master-0/data/instance:
uuid: "998fc82f37904949aa983176728483d2"
format_stamp: "Formatted at 2026-08-05 23:13:02 on dist-test-slave-ss4n"
server_key: "cac85564df6bb2d1e9e5c73796f7e7e2"
server_key_iv: "bf62f7997638a82fc16c8baf12fc098b"
server_key_version: "encryptionkey@0"
I20260805 23:13:02.374061 7572 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785971406251336-25971-0/minicluster-data/master-0/wal/instance:
uuid: "998fc82f37904949aa983176728483d2"
format_stamp: "Formatted at 2026-08-05 23:13:02 on dist-test-slave-ss4n"
server_key: "cac85564df6bb2d1e9e5c73796f7e7e2"
server_key_iv: "bf62f7997638a82fc16c8baf12fc098b"
server_key_version: "encryptionkey@0"
I20260805 23:13:02.378255 7572 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260805 23:13:02.381011 7587 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:13:02.382586 7572 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:13:02.382730 7572 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785971406251336-25971-0/minicluster-data/master-0/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785971406251336-25971-0/minicluster-data/master-0/wal
uuid: "998fc82f37904949aa983176728483d2"
format_stamp: "Formatted at 2026-08-05 23:13:02 on dist-test-slave-ss4n"
server_key: "cac85564df6bb2d1e9e5c73796f7e7e2"
server_key_iv: "bf62f7997638a82fc16c8baf12fc098b"
server_key_version: "encryptionkey@0"
I20260805 23:13:02.382853 7572 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785971406251336-25971-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785971406251336-25971-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785971406251336-25971-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:13:02.434120 7572 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:13:02.438759 7572 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:13:02.438949 7572 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:13:02.447184 7572 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.254:37317
I20260805 23:13:02.447213 7639 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.254:37317 every 8 connection(s)
I20260805 23:13:02.448220 7572 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
I20260805 23:13:02.451251 7640 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:13:02.457552 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 7572
I20260805 23:13:02.457792 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785971406251336-25971-0/minicluster-data/master-0/wal/instance
I20260805 23:13:02.457792 7640 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 998fc82f37904949aa983176728483d2: Bootstrap starting.
I20260805 23:13:02.458128 25971 external_mini_cluster.cc:1428] Setting key e0e27f4ef54198fbc3cfed1dbcddcdc8
I20260805 23:13:02.460855 7640 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 998fc82f37904949aa983176728483d2: Neither blocks nor log segments found. Creating new log.
I20260805 23:13:02.462384 7640 log.cc:824] T 00000000000000000000000000000000 P 998fc82f37904949aa983176728483d2: Log is configured to *not* fsync() on all Append() calls
I20260805 23:13:02.465271 7640 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 998fc82f37904949aa983176728483d2: No bootstrap required, opened a new log
Aug 05 23:13:02 dist-test-slave-ss4n krb5kdc[7558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971582, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM
I20260805 23:13:02.470978 7640 raft_consensus.cc:348] T 00000000000000000000000000000000 P 998fc82f37904949aa983176728483d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "998fc82f37904949aa983176728483d2" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 37317 } }
I20260805 23:13:02.471367 7640 raft_consensus.cc:374] T 00000000000000000000000000000000 P 998fc82f37904949aa983176728483d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:13:02.471436 7640 raft_consensus.cc:729] T 00000000000000000000000000000000 P 998fc82f37904949aa983176728483d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998fc82f37904949aa983176728483d2, State: Initialized, Role: FOLLOWER
I20260805 23:13:02.472245 7640 consensus_queue.cc:260] T 00000000000000000000000000000000 P 998fc82f37904949aa983176728483d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "998fc82f37904949aa983176728483d2" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 37317 } }
I20260805 23:13:02.472455 7640 raft_consensus.cc:388] T 00000000000000000000000000000000 P 998fc82f37904949aa983176728483d2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:13:02.472537 7640 raft_consensus.cc:482] T 00000000000000000000000000000000 P 998fc82f37904949aa983176728483d2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:13:02.472720 7640 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 998fc82f37904949aa983176728483d2 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:13:02.474232 7640 raft_consensus.cc:504] T 00000000000000000000000000000000 P 998fc82f37904949aa983176728483d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "998fc82f37904949aa983176728483d2" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 37317 } }
I20260805 23:13:02.474686 7640 leader_election.cc:302] T 00000000000000000000000000000000 P 998fc82f37904949aa983176728483d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 998fc82f37904949aa983176728483d2; no voters:
I20260805 23:13:02.475108 7640 leader_election.cc:288] T 00000000000000000000000000000000 P 998fc82f37904949aa983176728483d2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 23:13:02.475203 7645 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 998fc82f37904949aa983176728483d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:13:02.475417 7645 raft_consensus.cc:686] T 00000000000000000000000000000000 P 998fc82f37904949aa983176728483d2 [term 1 LEADER]: Becoming Leader. State: Replica: 998fc82f37904949aa983176728483d2, State: Running, Role: LEADER
I20260805 23:13:02.475837 7645 consensus_queue.cc:237] T 00000000000000000000000000000000 P 998fc82f37904949aa983176728483d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "998fc82f37904949aa983176728483d2" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 37317 } }
I20260805 23:13:02.476643 7640 sys_catalog.cc:564] T 00000000000000000000000000000000 P 998fc82f37904949aa983176728483d2 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:13:02.477769 7643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:02.459690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35590 (local address 127.25.92.254:37317)
0805 23:13:02.460267 (+ 577us) server_negotiation.cc:206] Beginning negotiation
0805 23:13:02.460284 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0805 23:13:02.460326 (+ 42us) server_negotiation.cc:407] Connection header received
0805 23:13:02.461445 (+ 1119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:13:02.461469 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:13:02.461921 (+ 452us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:13:02.462302 (+ 381us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:13:02.463491 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:13:02.465150 (+ 1659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:13:02.466317 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:13:02.467151 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:13:02.467364 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:13:02.470443 (+ 3079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:13:02.470451 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:13:02.470475 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:13:02.470516 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:13:02.473207 (+ 2691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:13:02.473899 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:13:02.473906 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:13:02.473913 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:13:02.474120 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:13:02.474563 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:13:02.474567 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:13:02.474569 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:13:02.475155 (+ 586us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:13:02.475440 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0805 23:13:02.476351 (+ 911us) server_negotiation.cc:299] Negotiation successful
0805 23:13:02.476646 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":398,"thread_start_us":161,"threads_started":1}
I20260805 23:13:02.478045 7647 sys_catalog.cc:455] T 00000000000000000000000000000000 P 998fc82f37904949aa983176728483d2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 998fc82f37904949aa983176728483d2. Latest consensus state: current_term: 1 leader_uuid: "998fc82f37904949aa983176728483d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "998fc82f37904949aa983176728483d2" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 37317 } } }
I20260805 23:13:02.478178 7647 sys_catalog.cc:458] T 00000000000000000000000000000000 P 998fc82f37904949aa983176728483d2 [sys.catalog]: This master's current role is: LEADER
I20260805 23:13:02.477756 7646 sys_catalog.cc:455] T 00000000000000000000000000000000 P 998fc82f37904949aa983176728483d2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "998fc82f37904949aa983176728483d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "998fc82f37904949aa983176728483d2" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 37317 } } }
I20260805 23:13:02.478473 7646 sys_catalog.cc:458] T 00000000000000000000000000000000 P 998fc82f37904949aa983176728483d2 [sys.catalog]: This master's current role is: LEADER
I20260805 23:13:02.478643 7654 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:13:02.482064 25971 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260805 23:13:02.482230 7654 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:13:02.488519 7654 catalog_manager.cc:1347] Generated new cluster ID: 118746b1cf254ecbb1135c124e6ef0b1
I20260805 23:13:02.488636 7654 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:13:02.499123 7654 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:13:02.500706 7654 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:13:02.512889 7654 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 998fc82f37904949aa983176728483d2: Generated new TSK 0
I20260805 23:13:02.513943 7654 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260805 23:13:02.539557 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 7572
2026-08-05T23:13:02Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1561 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 23:13:02 dist-test-slave-ss4n krb5kdc[7668](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 23:13:02 dist-test-slave-ss4n krb5kdc[7668](info): set up 2 sockets
Aug 05 23:13:02 dist-test-slave-ss4n krb5kdc[7668](info): commencing operation
krb5kdc: starting...
W20260805 23:13:03.696453 25971 mini_kdc.cc:121] Time spent starting KDC: real 1.134s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/krb5kdc/test-user.keytab.
Aug 05 23:13:03 dist-test-slave-ss4n krb5kdc[7668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971583, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T23:13:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:13:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.92.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.92.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.92.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab.
Entry for principal kudu/127.25.92.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab.
I20260805 23:13:03.842191 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.254:45961
--webserver_interface=127.25.92.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab
--principal=kudu/127.25.92.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:41983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.92.254:45961
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:13:03.960945 7681 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:13:03.961297 7681 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:13:03.961380 7681 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:13:03.965562 7681 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:13:03.965682 7681 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:13:03.965721 7681 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:13:03.965758 7681 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:13:03.965798 7681 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:13:03.972152 7681 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:41983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.92.254:45961
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab
--principal=kudu/127.25.92.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.254:45961
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.92.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7681
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:13:03.973362 7681 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:13:03.974480 7681 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:13:03.983814 7690 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:13:03.983884 7681 server_base.cc:1034] running on GCE node
W20260805 23:13:03.983791 7686 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:13:03.983816 7687 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:13:03.984823 7681 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:13:03.986168 7681 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:13:03.987407 7681 hybrid_clock.cc:648] HybridClock initialized: now 1785971583987363 us; error 60 us; skew 500 ppm
Aug 05 23:13:03 dist-test-slave-ss4n krb5kdc[7668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971583, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:13:03.995901 7681 init.cc:377] Logged in from keytab as kudu/127.25.92.254@KRBTEST.COM (short username kudu)
I20260805 23:13:03.997028 7681 webserver.cc:460] Webserver started at http://127.25.92.254:46373/ using document root <none> and password file <none>
I20260805 23:13:03.997730 7681 fs_manager.cc:359] Metadata directory not provided
I20260805 23:13:03.997829 7681 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:13:03.998078 7681 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:13:04.000339 7681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/master-0/data/instance:
uuid: "cea136ae38a74d03b8dfdfc98787079d"
format_stamp: "Formatted at 2026-08-05 23:13:03 on dist-test-slave-ss4n"
server_key: "f5eff3ff5f4038c6d9d0ec33d02b3020"
server_key_iv: "4787f4b860dd10e3df762e1ae1ff1bad"
server_key_version: "encryptionkey@0"
I20260805 23:13:04.000932 7681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/master-0/wal/instance:
uuid: "cea136ae38a74d03b8dfdfc98787079d"
format_stamp: "Formatted at 2026-08-05 23:13:03 on dist-test-slave-ss4n"
server_key: "f5eff3ff5f4038c6d9d0ec33d02b3020"
server_key_iv: "4787f4b860dd10e3df762e1ae1ff1bad"
server_key_version: "encryptionkey@0"
I20260805 23:13:04.005066 7681 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260805 23:13:04.007876 7697 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:13:04.009400 7681 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260805 23:13:04.009606 7681 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/master-0/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/master-0/wal
uuid: "cea136ae38a74d03b8dfdfc98787079d"
format_stamp: "Formatted at 2026-08-05 23:13:03 on dist-test-slave-ss4n"
server_key: "f5eff3ff5f4038c6d9d0ec33d02b3020"
server_key_iv: "4787f4b860dd10e3df762e1ae1ff1bad"
server_key_version: "encryptionkey@0"
I20260805 23:13:04.009756 7681 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:13:04.035413 7681 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:13:04.039602 7681 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:13:04.039940 7681 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:13:04.048136 7681 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.254:45961
I20260805 23:13:04.048492 7751 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.254:45961 every 8 connection(s)
I20260805 23:13:04.049255 7681 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
I20260805 23:13:04.049939 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 7681
I20260805 23:13:04.050091 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/master-0/wal/instance
I20260805 23:13:04.050374 25971 external_mini_cluster.cc:1428] Setting key dfc5d9d5756a12ecf3fac619fa011a0a
I20260805 23:13:04.052752 7752 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:13:04.059969 7752 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cea136ae38a74d03b8dfdfc98787079d: Bootstrap starting.
Aug 05 23:13:04 dist-test-slave-ss4n krb5kdc[7668](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971583, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM
I20260805 23:13:04.062508 7752 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cea136ae38a74d03b8dfdfc98787079d: Neither blocks nor log segments found. Creating new log.
I20260805 23:13:04.063285 7752 log.cc:824] T 00000000000000000000000000000000 P cea136ae38a74d03b8dfdfc98787079d: Log is configured to *not* fsync() on all Append() calls
I20260805 23:13:04.065433 7752 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cea136ae38a74d03b8dfdfc98787079d: No bootstrap required, opened a new log
I20260805 23:13:04.065877 7755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:04.051926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48536 (local address 127.25.92.254:45961)
0805 23:13:04.052412 (+ 486us) server_negotiation.cc:206] Beginning negotiation
0805 23:13:04.052436 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0805 23:13:04.052471 (+ 35us) server_negotiation.cc:407] Connection header received
0805 23:13:04.053197 (+ 726us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:13:04.053216 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:13:04.053618 (+ 402us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:13:04.053910 (+ 292us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:13:04.054871 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:13:04.055906 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:13:04.057084 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:13:04.057680 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:13:04.057804 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:13:04.060791 (+ 2987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:13:04.060796 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:13:04.060809 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:13:04.060841 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:13:04.062832 (+ 1991us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:13:04.063327 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:13:04.063333 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:13:04.063338 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:13:04.063496 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:13:04.063802 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:13:04.063805 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:13:04.063807 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:13:04.064230 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:13:04.064436 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0805 23:13:04.064765 (+ 329us) server_negotiation.cc:299] Negotiation successful
0805 23:13:04.065067 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":190,"threads_started":1}
I20260805 23:13:04.069684 7752 raft_consensus.cc:348] T 00000000000000000000000000000000 P cea136ae38a74d03b8dfdfc98787079d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cea136ae38a74d03b8dfdfc98787079d" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 45961 } }
I20260805 23:13:04.069926 7752 raft_consensus.cc:374] T 00000000000000000000000000000000 P cea136ae38a74d03b8dfdfc98787079d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:13:04.069962 7752 raft_consensus.cc:729] T 00000000000000000000000000000000 P cea136ae38a74d03b8dfdfc98787079d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cea136ae38a74d03b8dfdfc98787079d, State: Initialized, Role: FOLLOWER
I20260805 23:13:04.070467 7752 consensus_queue.cc:260] T 00000000000000000000000000000000 P cea136ae38a74d03b8dfdfc98787079d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cea136ae38a74d03b8dfdfc98787079d" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 45961 } }
I20260805 23:13:04.070623 7752 raft_consensus.cc:388] T 00000000000000000000000000000000 P cea136ae38a74d03b8dfdfc98787079d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:13:04.070709 7752 raft_consensus.cc:482] T 00000000000000000000000000000000 P cea136ae38a74d03b8dfdfc98787079d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:13:04.070785 7752 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cea136ae38a74d03b8dfdfc98787079d [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:13:04.071911 7752 raft_consensus.cc:504] T 00000000000000000000000000000000 P cea136ae38a74d03b8dfdfc98787079d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cea136ae38a74d03b8dfdfc98787079d" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 45961 } }
I20260805 23:13:04.072288 7752 leader_election.cc:302] T 00000000000000000000000000000000 P cea136ae38a74d03b8dfdfc98787079d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cea136ae38a74d03b8dfdfc98787079d; no voters:
I20260805 23:13:04.072662 7752 leader_election.cc:288] T 00000000000000000000000000000000 P cea136ae38a74d03b8dfdfc98787079d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 23:13:04.073024 7757 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cea136ae38a74d03b8dfdfc98787079d [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:13:04.073324 7757 raft_consensus.cc:686] T 00000000000000000000000000000000 P cea136ae38a74d03b8dfdfc98787079d [term 1 LEADER]: Becoming Leader. State: Replica: cea136ae38a74d03b8dfdfc98787079d, State: Running, Role: LEADER
I20260805 23:13:04.073850 7757 consensus_queue.cc:237] T 00000000000000000000000000000000 P cea136ae38a74d03b8dfdfc98787079d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cea136ae38a74d03b8dfdfc98787079d" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 45961 } }
I20260805 23:13:04.074121 7752 sys_catalog.cc:564] T 00000000000000000000000000000000 P cea136ae38a74d03b8dfdfc98787079d [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:13:04.076267 7758 sys_catalog.cc:455] T 00000000000000000000000000000000 P cea136ae38a74d03b8dfdfc98787079d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cea136ae38a74d03b8dfdfc98787079d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cea136ae38a74d03b8dfdfc98787079d" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 45961 } } }
I20260805 23:13:04.076447 7758 sys_catalog.cc:458] T 00000000000000000000000000000000 P cea136ae38a74d03b8dfdfc98787079d [sys.catalog]: This master's current role is: LEADER
I20260805 23:13:04.076207 7759 sys_catalog.cc:455] T 00000000000000000000000000000000 P cea136ae38a74d03b8dfdfc98787079d [sys.catalog]: SysCatalogTable state changed. Reason: New leader cea136ae38a74d03b8dfdfc98787079d. Latest consensus state: current_term: 1 leader_uuid: "cea136ae38a74d03b8dfdfc98787079d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cea136ae38a74d03b8dfdfc98787079d" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 45961 } } }
I20260805 23:13:04.076538 7759 sys_catalog.cc:458] T 00000000000000000000000000000000 P cea136ae38a74d03b8dfdfc98787079d [sys.catalog]: This master's current role is: LEADER
W20260805 23:13:04.080084 7772 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cea136ae38a74d03b8dfdfc98787079d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260805 23:13:04.080207 7772 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260805 23:13:04.080317 7773 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:13:04.081318 7773 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:13:04.087643 7773 catalog_manager.cc:1347] Generated new cluster ID: 31a5fac73c5b4a22a381a0c99d3ea80a
I20260805 23:13:04.087731 7773 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:13:04.103312 7773 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:13:04.104940 7773 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:13:04.116930 7773 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cea136ae38a74d03b8dfdfc98787079d: Generated new TSK 0
I20260805 23:13:04.117928 7773 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.92.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.92.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.92.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.193.keytab.
Entry for principal kudu/127.25.92.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.193.keytab.
I20260805 23:13:04.160602 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.92.193:0
--local_ip_for_outbound_sockets=127.25.92.193
--webserver_interface=127.25.92.193
--webserver_port=0
--tserver_master_addrs=127.25.92.254:45961
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.193.keytab
--principal=kudu/127.25.92.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:41983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:13:04.280731 7779 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:13:04.281023 7779 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:13:04.281067 7779 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:13:04.285260 7779 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:13:04.285343 7779 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:13:04.285441 7779 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.92.193
I20260805 23:13:04.291290 7779 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:41983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.193.keytab
--principal=kudu/127.25.92.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.92.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.92.193
--webserver_port=0
--tserver_master_addrs=127.25.92.254:45961
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7779
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.92.193
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:13:04.292696 7779 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:13:04.293784 7779 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:13:04.304436 7787 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:13:04.304494 7779 server_base.cc:1034] running on GCE node
W20260805 23:13:04.304479 7785 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:13:04.304519 7784 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:13:04.305418 7779 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:13:04.306237 7779 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:13:04.307406 7779 hybrid_clock.cc:648] HybridClock initialized: now 1785971584307390 us; error 32 us; skew 500 ppm
Aug 05 23:13:04 dist-test-slave-ss4n krb5kdc[7668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971584, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:13:04.310967 7779 init.cc:377] Logged in from keytab as kudu/127.25.92.193@KRBTEST.COM (short username kudu)
I20260805 23:13:04.312098 7779 webserver.cc:460] Webserver started at http://127.25.92.193:36179/ using document root <none> and password file <none>
I20260805 23:13:04.312731 7779 fs_manager.cc:359] Metadata directory not provided
I20260805 23:13:04.312801 7779 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:13:04.313060 7779 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:13:04.315635 7779 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-0/data/instance:
uuid: "9d7053e2a49447dc8f347c9188078e94"
format_stamp: "Formatted at 2026-08-05 23:13:04 on dist-test-slave-ss4n"
server_key: "d5e2b7c396a15bd3a8b77add6d817f32"
server_key_iv: "f0c4bb51274444f28601105a60511d8d"
server_key_version: "encryptionkey@0"
I20260805 23:13:04.316547 7779 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-0/wal/instance:
uuid: "9d7053e2a49447dc8f347c9188078e94"
format_stamp: "Formatted at 2026-08-05 23:13:04 on dist-test-slave-ss4n"
server_key: "d5e2b7c396a15bd3a8b77add6d817f32"
server_key_iv: "f0c4bb51274444f28601105a60511d8d"
server_key_version: "encryptionkey@0"
I20260805 23:13:04.321254 7779 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260805 23:13:04.324276 7794 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:13:04.326071 7779 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:13:04.326277 7779 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-0/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-0/wal
uuid: "9d7053e2a49447dc8f347c9188078e94"
format_stamp: "Formatted at 2026-08-05 23:13:04 on dist-test-slave-ss4n"
server_key: "d5e2b7c396a15bd3a8b77add6d817f32"
server_key_iv: "f0c4bb51274444f28601105a60511d8d"
server_key_version: "encryptionkey@0"
I20260805 23:13:04.326435 7779 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:13:04.355695 7779 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:13:04.359648 7779 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:13:04.359894 7779 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:13:04.360570 7779 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:13:04.361589 7779 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:13:04.361663 7779 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:13:04.361744 7779 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:13:04.361800 7779 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:13:04.372047 7779 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.193:43069
I20260805 23:13:04.372073 7911 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.193:43069 every 8 connection(s)
I20260805 23:13:04.373075 7779 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-0/data/info.pb
I20260805 23:13:04.379225 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 7779
I20260805 23:13:04.379406 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-0/wal/instance
I20260805 23:13:04.379871 25971 external_mini_cluster.cc:1428] Setting key ffc89de9bc8b71f9829d50f747ab5518
Aug 05 23:13:04 dist-test-slave-ss4n krb5kdc[7668](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971584, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.193@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM
I20260805 23:13:04.390511 7755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:04.375258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.193:34695 (local address 127.25.92.254:45961)
0805 23:13:04.375435 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0805 23:13:04.375513 (+ 78us) server_negotiation.cc:399] Waiting for connection header
0805 23:13:04.376624 (+ 1111us) server_negotiation.cc:407] Connection header received
0805 23:13:04.378155 (+ 1531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:13:04.378159 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:13:04.378213 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:13:04.378307 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:13:04.379684 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:13:04.380487 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:13:04.381891 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:13:04.382346 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:13:04.382485 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:13:04.385651 (+ 3166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:13:04.385658 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:13:04.385664 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:13:04.385703 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:13:04.387722 (+ 2019us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:13:04.388362 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:13:04.388366 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:13:04.388368 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:13:04.388479 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:13:04.388992 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:13:04.388997 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:13:04.388999 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:13:04.389307 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:13:04.389429 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0805 23:13:04.390105 (+ 676us) server_negotiation.cc:299] Negotiation successful
0805 23:13:04.390267 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
WARNING: no policy specified for kudu/127.25.92.194@KRBTEST.COM; defaulting to no policy
I20260805 23:13:04.391352 7914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:04.375817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:45961 (local address 127.25.92.193:34695)
0805 23:13:04.376453 (+ 636us) negotiation.cc:107] Waiting for socket to connect
0805 23:13:04.376501 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0805 23:13:04.377904 (+ 1403us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:13:04.378489 (+ 585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:13:04.378496 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:13:04.378922 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:13:04.379489 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:13:04.379506 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:13:04.380630 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:13:04.380634 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:13:04.381762 (+ 1128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:13:04.381772 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:13:04.382522 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:13:04.382525 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:13:04.382690 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:13:04.383454 (+ 764us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:13:04.383483 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:13:04.385385 (+ 1902us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:13:04.387886 (+ 2501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:13:04.387892 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:13:04.387907 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:13:04.388230 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:13:04.388585 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:13:04.388588 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:13:04.388591 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:13:04.388740 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:13:04.389463 (+ 723us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:13:04.389469 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:13:04.389815 (+ 346us) client_negotiation.cc:769] Sending connection context
0805 23:13:04.389988 (+ 173us) client_negotiation.cc:240] Negotiation successful
0805 23:13:04.390291 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":146,"threads_started":1}
I20260805 23:13:04.393031 7912 heartbeater.cc:347] Connected to a master server at 127.25.92.254:45961
I20260805 23:13:04.393390 7912 heartbeater.cc:464] Registering TS with master...
I20260805 23:13:04.394088 7912 heartbeater.cc:511] Master 127.25.92.254:45961 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.92.194@KRBTEST.COM" created.
I20260805 23:13:04.396024 7715 ts_manager.cc:194] Registered new tserver with Master: 9d7053e2a49447dc8f347c9188078e94 (127.25.92.193:43069)
I20260805 23:13:04.397276 7715 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.92.193@KRBTEST.COM'} at 127.25.92.193:34695
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.92.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.194.keytab.
Entry for principal kudu/127.25.92.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.194.keytab.
I20260805 23:13:04.414016 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.92.194:0
--local_ip_for_outbound_sockets=127.25.92.194
--webserver_interface=127.25.92.194
--webserver_port=0
--tserver_master_addrs=127.25.92.254:45961
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.194.keytab
--principal=kudu/127.25.92.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:41983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:13:04.539032 7917 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:13:04.539394 7917 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:13:04.539475 7917 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:13:04.544028 7917 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:13:04.544139 7917 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:13:04.544241 7917 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.92.194
I20260805 23:13:04.549778 7917 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:41983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.194.keytab
--principal=kudu/127.25.92.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.92.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.92.194
--webserver_port=0
--tserver_master_addrs=127.25.92.254:45961
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7917
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.92.194
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:13:04.550966 7917 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:13:04.552016 7917 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:13:04.562314 7923 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:13:04.562304 7924 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:13:04.562336 7926 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:13:04.562748 7917 server_base.cc:1034] running on GCE node
I20260805 23:13:04.563222 7917 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:13:04.563956 7917 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:13:04.565182 7917 hybrid_clock.cc:648] HybridClock initialized: now 1785971584565153 us; error 47 us; skew 500 ppm
Aug 05 23:13:04 dist-test-slave-ss4n krb5kdc[7668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971584, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:13:04.568220 7917 init.cc:377] Logged in from keytab as kudu/127.25.92.194@KRBTEST.COM (short username kudu)
I20260805 23:13:04.569185 7917 webserver.cc:460] Webserver started at http://127.25.92.194:44697/ using document root <none> and password file <none>
I20260805 23:13:04.569871 7917 fs_manager.cc:359] Metadata directory not provided
I20260805 23:13:04.569942 7917 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:13:04.570120 7917 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:13:04.572490 7917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-1/data/instance:
uuid: "3f3cc8ff1e664a859a891b80666307f1"
format_stamp: "Formatted at 2026-08-05 23:13:04 on dist-test-slave-ss4n"
server_key: "0831a59f62202ce72c947f5a4bdb6bc4"
server_key_iv: "803573aa0d560547b97631878b70eb79"
server_key_version: "encryptionkey@0"
I20260805 23:13:04.573060 7917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-1/wal/instance:
uuid: "3f3cc8ff1e664a859a891b80666307f1"
format_stamp: "Formatted at 2026-08-05 23:13:04 on dist-test-slave-ss4n"
server_key: "0831a59f62202ce72c947f5a4bdb6bc4"
server_key_iv: "803573aa0d560547b97631878b70eb79"
server_key_version: "encryptionkey@0"
I20260805 23:13:04.577442 7917 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260805 23:13:04.580157 7933 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:13:04.581558 7917 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260805 23:13:04.581665 7917 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-1/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-1/wal
uuid: "3f3cc8ff1e664a859a891b80666307f1"
format_stamp: "Formatted at 2026-08-05 23:13:04 on dist-test-slave-ss4n"
server_key: "0831a59f62202ce72c947f5a4bdb6bc4"
server_key_iv: "803573aa0d560547b97631878b70eb79"
server_key_version: "encryptionkey@0"
I20260805 23:13:04.581763 7917 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:13:04.600379 7917 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:13:04.604063 7917 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:13:04.604233 7917 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:13:04.604806 7917 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:13:04.605692 7917 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:13:04.605744 7917 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:13:04.605787 7917 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:13:04.605803 7917 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:13:04.615618 7917 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.194:37313
I20260805 23:13:04.615640 8049 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.194:37313 every 8 connection(s)
I20260805 23:13:04.616612 7917 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-1/data/info.pb
I20260805 23:13:04.622500 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 7917
I20260805 23:13:04.622656 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-1/wal/instance
I20260805 23:13:04.622989 25971 external_mini_cluster.cc:1428] Setting key 221b8fb5480a06cd06be557061f141ee
Aug 05 23:13:04 dist-test-slave-ss4n krb5kdc[7668](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971584, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.194@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM
I20260805 23:13:04.634542 7755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:04.618806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.194:57087 (local address 127.25.92.254:45961)
0805 23:13:04.619003 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0805 23:13:04.619008 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:13:04.619958 (+ 950us) server_negotiation.cc:407] Connection header received
0805 23:13:04.621338 (+ 1380us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:13:04.621340 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:13:04.621393 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:13:04.621526 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:13:04.623012 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:13:04.623741 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:13:04.625421 (+ 1680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:13:04.625874 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:13:04.626004 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:13:04.629323 (+ 3319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:13:04.629331 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:13:04.629338 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:13:04.629382 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:13:04.631777 (+ 2395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:13:04.632459 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:13:04.632466 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:13:04.632470 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:13:04.632598 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:13:04.633020 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:13:04.633024 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:13:04.633026 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:13:04.633317 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:13:04.633449 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0805 23:13:04.634115 (+ 666us) server_negotiation.cc:299] Negotiation successful
0805 23:13:04.634276 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
WARNING: no policy specified for kudu/127.25.92.195@KRBTEST.COM; defaulting to no policy
I20260805 23:13:04.635298 8052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:04.619250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:45961 (local address 127.25.92.194:57087)
0805 23:13:04.619784 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0805 23:13:04.619832 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0805 23:13:04.621105 (+ 1273us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:13:04.621737 (+ 632us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:13:04.621745 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:13:04.622170 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:13:04.622844 (+ 674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:13:04.622862 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:13:04.623960 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:13:04.623964 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:13:04.625271 (+ 1307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:13:04.625283 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:13:04.626050 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:13:04.626054 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:13:04.626211 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:13:04.626959 (+ 748us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:13:04.626989 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:13:04.629059 (+ 2070us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:13:04.631952 (+ 2893us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:13:04.631959 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:13:04.631977 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:13:04.632321 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:13:04.632730 (+ 409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:13:04.632732 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:13:04.632734 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:13:04.632902 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:13:04.633443 (+ 541us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:13:04.633450 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:13:04.633819 (+ 369us) client_negotiation.cc:769] Sending connection context
0805 23:13:04.633999 (+ 180us) client_negotiation.cc:240] Negotiation successful
0805 23:13:04.634295 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":134,"threads_started":1}
I20260805 23:13:04.636973 8050 heartbeater.cc:347] Connected to a master server at 127.25.92.254:45961
I20260805 23:13:04.637264 8050 heartbeater.cc:464] Registering TS with master...
I20260805 23:13:04.637902 8050 heartbeater.cc:511] Master 127.25.92.254:45961 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.92.195@KRBTEST.COM" created.
I20260805 23:13:04.639142 7715 ts_manager.cc:194] Registered new tserver with Master: 3f3cc8ff1e664a859a891b80666307f1 (127.25.92.194:37313)
I20260805 23:13:04.640038 7715 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.92.194@KRBTEST.COM'} at 127.25.92.194:57087
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.92.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.195.keytab.
Entry for principal kudu/127.25.92.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.195.keytab.
I20260805 23:13:04.658563 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.92.195:0
--local_ip_for_outbound_sockets=127.25.92.195
--webserver_interface=127.25.92.195
--webserver_port=0
--tserver_master_addrs=127.25.92.254:45961
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.195.keytab
--principal=kudu/127.25.92.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:41983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:13:04.777864 8055 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:13:04.778194 8055 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:13:04.778277 8055 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:13:04.782567 8055 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:13:04.782667 8055 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:13:04.782759 8055 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.92.195
I20260805 23:13:04.788533 8055 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:41983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.195.keytab
--principal=kudu/127.25.92.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.92.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.92.195
--webserver_port=0
--tserver_master_addrs=127.25.92.254:45961
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8055
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.92.195
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:13:04.789772 8055 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:13:04.790828 8055 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:13:04.801255 8060 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:13:04.801312 8055 server_base.cc:1034] running on GCE node
W20260805 23:13:04.801200 8061 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:13:04.801213 8063 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:13:04.802078 8055 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:13:04.802841 8055 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:13:04.804070 8055 hybrid_clock.cc:648] HybridClock initialized: now 1785971584804028 us; error 51 us; skew 500 ppm
Aug 05 23:13:04 dist-test-slave-ss4n krb5kdc[7668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971584, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:13:04.807273 8055 init.cc:377] Logged in from keytab as kudu/127.25.92.195@KRBTEST.COM (short username kudu)
I20260805 23:13:04.808254 8055 webserver.cc:460] Webserver started at http://127.25.92.195:44611/ using document root <none> and password file <none>
I20260805 23:13:04.808866 8055 fs_manager.cc:359] Metadata directory not provided
I20260805 23:13:04.808962 8055 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:13:04.809216 8055 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:13:04.811403 8055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-2/data/instance:
uuid: "57ca592f72ed4d6f8fd9a07643f02fd5"
format_stamp: "Formatted at 2026-08-05 23:13:04 on dist-test-slave-ss4n"
server_key: "6148b95a70b48af2769e9dff78e44ce4"
server_key_iv: "27d0d2bb33804600190f62945429256a"
server_key_version: "encryptionkey@0"
I20260805 23:13:04.812013 8055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-2/wal/instance:
uuid: "57ca592f72ed4d6f8fd9a07643f02fd5"
format_stamp: "Formatted at 2026-08-05 23:13:04 on dist-test-slave-ss4n"
server_key: "6148b95a70b48af2769e9dff78e44ce4"
server_key_iv: "27d0d2bb33804600190f62945429256a"
server_key_version: "encryptionkey@0"
I20260805 23:13:04.816298 8055 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260805 23:13:04.819267 8070 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:13:04.820757 8055 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260805 23:13:04.820927 8055 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-2/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-2/wal
uuid: "57ca592f72ed4d6f8fd9a07643f02fd5"
format_stamp: "Formatted at 2026-08-05 23:13:04 on dist-test-slave-ss4n"
server_key: "6148b95a70b48af2769e9dff78e44ce4"
server_key_iv: "27d0d2bb33804600190f62945429256a"
server_key_version: "encryptionkey@0"
I20260805 23:13:04.821092 8055 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:13:04.838617 8055 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:13:04.842418 8055 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:13:04.842653 8055 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:13:04.843293 8055 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:13:04.844172 8055 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:13:04.844219 8055 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:13:04.844288 8055 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:13:04.844328 8055 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:13:04.853911 8055 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.195:38153
I20260805 23:13:04.854012 8189 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.195:38153 every 8 connection(s)
I20260805 23:13:04.854938 8055 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-2/data/info.pb
I20260805 23:13:04.856503 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 8055
I20260805 23:13:04.856633 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-2/wal/instance
I20260805 23:13:04.857024 25971 external_mini_cluster.cc:1428] Setting key 4b6293705a9ea0d85cb4b7d552ce66ce
Aug 05 23:13:04 dist-test-slave-ss4n krb5kdc[7668](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971584, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.195@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM
I20260805 23:13:04.870131 7755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:04.857016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.195:35049 (local address 127.25.92.254:45961)
0805 23:13:04.857224 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0805 23:13:04.857229 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:13:04.858320 (+ 1091us) server_negotiation.cc:407] Connection header received
0805 23:13:04.860079 (+ 1759us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:13:04.860082 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:13:04.860140 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:13:04.860218 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:13:04.861540 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:13:04.862110 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:13:04.863302 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:13:04.863542 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:13:04.863666 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:13:04.866443 (+ 2777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:13:04.866446 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:13:04.866448 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:13:04.866475 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:13:04.867856 (+ 1381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:13:04.868425 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:13:04.868434 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:13:04.868435 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:13:04.868513 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:13:04.868883 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:13:04.868887 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:13:04.868889 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:13:04.869090 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:13:04.869205 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0805 23:13:04.869820 (+ 615us) server_negotiation.cc:299] Negotiation successful
0805 23:13:04.869961 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260805 23:13:04.871140 8193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:04.857473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:45961 (local address 127.25.92.195:35049)
0805 23:13:04.858131 (+ 658us) negotiation.cc:107] Waiting for socket to connect
0805 23:13:04.858215 (+ 84us) client_negotiation.cc:174] Beginning negotiation
0805 23:13:04.859855 (+ 1640us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:13:04.860400 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:13:04.860407 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:13:04.860865 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:13:04.861372 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:13:04.861384 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:13:04.862269 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:13:04.862272 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:13:04.863172 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:13:04.863181 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:13:04.863707 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:13:04.863711 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:13:04.863889 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:13:04.864468 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:13:04.864485 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:13:04.866245 (+ 1760us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:13:04.867989 (+ 1744us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:13:04.867995 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:13:04.868013 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:13:04.868304 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:13:04.868619 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:13:04.868622 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:13:04.868624 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:13:04.868780 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:13:04.869211 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:13:04.869220 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:13:04.869554 (+ 334us) client_negotiation.cc:769] Sending connection context
0805 23:13:04.869766 (+ 212us) client_negotiation.cc:240] Negotiation successful
0805 23:13:04.870125 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":158,"threads_started":1}
I20260805 23:13:04.872277 8190 heartbeater.cc:347] Connected to a master server at 127.25.92.254:45961
I20260805 23:13:04.872498 8190 heartbeater.cc:464] Registering TS with master...
I20260805 23:13:04.872957 8190 heartbeater.cc:511] Master 127.25.92.254:45961 requested a full tablet report, sending...
I20260805 23:13:04.873961 7715 ts_manager.cc:194] Registered new tserver with Master: 57ca592f72ed4d6f8fd9a07643f02fd5 (127.25.92.195:38153)
I20260805 23:13:04.874630 7715 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.92.195@KRBTEST.COM'} at 127.25.92.195:35049
I20260805 23:13:04.883752 25971 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260805 23:13:04.894891 7755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:04.886205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48550 (local address 127.25.92.254:45961)
0805 23:13:04.886346 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0805 23:13:04.886349 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 23:13:04.886606 (+ 257us) server_negotiation.cc:407] Connection header received
0805 23:13:04.886734 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:13:04.886737 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:13:04.886784 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:13:04.886856 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:13:04.887642 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:13:04.888168 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:13:04.889211 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:13:04.889429 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:13:04.889602 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:13:04.891168 (+ 1566us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:13:04.891171 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:13:04.891173 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:13:04.891198 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:13:04.892960 (+ 1762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:13:04.893612 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:13:04.893615 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:13:04.893617 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:13:04.893718 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:13:04.894005 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:13:04.894007 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:13:04.894008 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:13:04.894246 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:13:04.894355 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0805 23:13:04.894540 (+ 185us) server_negotiation.cc:299] Negotiation successful
0805 23:13:04.894652 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260805 23:13:04.898488 7715 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48550:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260805 23:13:04.901927 7715 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260805 23:13:04.918085 8205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:04.911521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59332 (local address 127.25.92.195:38153)
0805 23:13:04.912311 (+ 790us) server_negotiation.cc:206] Beginning negotiation
0805 23:13:04.912324 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0805 23:13:04.912389 (+ 65us) server_negotiation.cc:407] Connection header received
0805 23:13:04.912652 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:13:04.912658 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:13:04.912888 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:13:04.913095 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:13:04.913783 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:13:04.914791 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:13:04.916388 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:13:04.917266 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:13:04.917389 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:13:04.917595 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0805 23:13:04.917800 (+ 205us) server_negotiation.cc:299] Negotiation successful
0805 23:13:04.917923 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":676,"thread_start_us":183,"threads_started":1}
I20260805 23:13:04.918087 8206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:04.911259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.195:38153 (local address 127.0.0.1:59332)
0805 23:13:04.912309 (+ 1050us) negotiation.cc:107] Waiting for socket to connect
0805 23:13:04.912322 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0805 23:13:04.912457 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:13:04.913287 (+ 830us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:13:04.913293 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:13:04.913329 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:13:04.913632 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:13:04.913640 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:13:04.914928 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:13:04.914935 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:13:04.916250 (+ 1315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:13:04.916260 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:13:04.917423 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:13:04.917426 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:13:04.917570 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:13:04.917613 (+ 43us) client_negotiation.cc:769] Sending connection context
0805 23:13:04.917752 (+ 139us) client_negotiation.cc:240] Negotiation successful
0805 23:13:04.917889 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":977,"mutex_wait_us":247,"spinlock_wait_cycles":18688,"thread_start_us":81,"threads_started":1}
I20260805 23:13:04.918480 8207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:04.912628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43056 (local address 127.25.92.193:43069)
0805 23:13:04.913096 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0805 23:13:04.913104 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0805 23:13:04.913331 (+ 227us) server_negotiation.cc:407] Connection header received
0805 23:13:04.913646 (+ 315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:13:04.913651 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:13:04.913866 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:13:04.914023 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:13:04.914323 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:13:04.915425 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:13:04.916788 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:13:04.917411 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:13:04.917558 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:13:04.917715 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0805 23:13:04.918199 (+ 484us) server_negotiation.cc:299] Negotiation successful
0805 23:13:04.918330 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":77,"threads_started":1}
I20260805 23:13:04.918625 8204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:04.911367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54820 (local address 127.25.92.194:37313)
0805 23:13:04.912110 (+ 743us) server_negotiation.cc:206] Beginning negotiation
0805 23:13:04.912122 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0805 23:13:04.912328 (+ 206us) server_negotiation.cc:407] Connection header received
0805 23:13:04.912644 (+ 316us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:13:04.912697 (+ 53us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:13:04.912894 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:13:04.913081 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:13:04.913765 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:13:04.915035 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:13:04.916581 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:13:04.917369 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:13:04.917463 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:13:04.917604 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0805 23:13:04.918172 (+ 568us) server_negotiation.cc:299] Negotiation successful
0805 23:13:04.918459 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":636,"thread_start_us":157,"threads_started":1}
I20260805 23:13:04.918102 8203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:04.911256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.194:37313 (local address 127.0.0.1:54820)
0805 23:13:04.912175 (+ 919us) negotiation.cc:107] Waiting for socket to connect
0805 23:13:04.912235 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0805 23:13:04.912457 (+ 222us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:13:04.913295 (+ 838us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:13:04.913297 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:13:04.913327 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:13:04.913632 (+ 305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:13:04.913640 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:13:04.915209 (+ 1569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:13:04.915212 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:13:04.916450 (+ 1238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:13:04.916460 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:13:04.917686 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:13:04.917689 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:13:04.917802 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:13:04.917827 (+ 25us) client_negotiation.cc:769] Sending connection context
0805 23:13:04.917902 (+ 75us) client_negotiation.cc:240] Negotiation successful
0805 23:13:04.917967 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":732,"mutex_wait_us":104,"spinlock_wait_cycles":6272,"thread_start_us":106,"threads_started":1}
I20260805 23:13:04.918116 8202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:04.912050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.193:43069 (local address 127.0.0.1:43056)
0805 23:13:04.913228 (+ 1178us) negotiation.cc:107] Waiting for socket to connect
0805 23:13:04.913239 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0805 23:13:04.913361 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:13:04.914006 (+ 645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:13:04.914009 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:13:04.914034 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:13:04.914182 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:13:04.914190 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:13:04.915554 (+ 1364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:13:04.915557 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:13:04.916669 (+ 1112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:13:04.916676 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:13:04.917781 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:13:04.917784 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:13:04.917865 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:13:04.917885 (+ 20us) client_negotiation.cc:769] Sending connection context
0805 23:13:04.917952 (+ 67us) client_negotiation.cc:240] Negotiation successful
0805 23:13:04.918015 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1117,"thread_start_us":119,"threads_started":1}
I20260805 23:13:04.921420 7982 tablet_service.cc:1467] Processing CreateTablet for tablet 010a96d4aeb243df9570dd9adf632526 (DEFAULT_TABLE table=test-table [id=a4d0cd17e8314dcc866a577685d67a66]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:13:04.921389 8122 tablet_service.cc:1467] Processing CreateTablet for tablet 010a96d4aeb243df9570dd9adf632526 (DEFAULT_TABLE table=test-table [id=a4d0cd17e8314dcc866a577685d67a66]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:13:04.922598 8122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 010a96d4aeb243df9570dd9adf632526. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:13:04.922600 7982 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 010a96d4aeb243df9570dd9adf632526. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:13:04.925024 7843 tablet_service.cc:1467] Processing CreateTablet for tablet 010a96d4aeb243df9570dd9adf632526 (DEFAULT_TABLE table=test-table [id=a4d0cd17e8314dcc866a577685d67a66]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:13:04.926303 7843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 010a96d4aeb243df9570dd9adf632526. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:13:04.929678 8209 tablet_bootstrap.cc:492] T 010a96d4aeb243df9570dd9adf632526 P 57ca592f72ed4d6f8fd9a07643f02fd5: Bootstrap starting.
I20260805 23:13:04.932195 8209 tablet_bootstrap.cc:654] T 010a96d4aeb243df9570dd9adf632526 P 57ca592f72ed4d6f8fd9a07643f02fd5: Neither blocks nor log segments found. Creating new log.
I20260805 23:13:04.933348 8209 log.cc:824] T 010a96d4aeb243df9570dd9adf632526 P 57ca592f72ed4d6f8fd9a07643f02fd5: Log is configured to *not* fsync() on all Append() calls
I20260805 23:13:04.934114 8211 tablet_bootstrap.cc:492] T 010a96d4aeb243df9570dd9adf632526 P 9d7053e2a49447dc8f347c9188078e94: Bootstrap starting.
I20260805 23:13:04.934171 8210 tablet_bootstrap.cc:492] T 010a96d4aeb243df9570dd9adf632526 P 3f3cc8ff1e664a859a891b80666307f1: Bootstrap starting.
I20260805 23:13:04.936429 8209 tablet_bootstrap.cc:492] T 010a96d4aeb243df9570dd9adf632526 P 57ca592f72ed4d6f8fd9a07643f02fd5: No bootstrap required, opened a new log
I20260805 23:13:04.936540 8211 tablet_bootstrap.cc:654] T 010a96d4aeb243df9570dd9adf632526 P 9d7053e2a49447dc8f347c9188078e94: Neither blocks nor log segments found. Creating new log.
I20260805 23:13:04.936656 8209 ts_tablet_manager.cc:1397] T 010a96d4aeb243df9570dd9adf632526 P 57ca592f72ed4d6f8fd9a07643f02fd5: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260805 23:13:04.937515 8211 log.cc:824] T 010a96d4aeb243df9570dd9adf632526 P 9d7053e2a49447dc8f347c9188078e94: Log is configured to *not* fsync() on all Append() calls
I20260805 23:13:04.938645 8210 tablet_bootstrap.cc:654] T 010a96d4aeb243df9570dd9adf632526 P 3f3cc8ff1e664a859a891b80666307f1: Neither blocks nor log segments found. Creating new log.
I20260805 23:13:04.939401 8210 log.cc:824] T 010a96d4aeb243df9570dd9adf632526 P 3f3cc8ff1e664a859a891b80666307f1: Log is configured to *not* fsync() on all Append() calls
I20260805 23:13:04.939883 8211 tablet_bootstrap.cc:492] T 010a96d4aeb243df9570dd9adf632526 P 9d7053e2a49447dc8f347c9188078e94: No bootstrap required, opened a new log
I20260805 23:13:04.940105 8211 ts_tablet_manager.cc:1397] T 010a96d4aeb243df9570dd9adf632526 P 9d7053e2a49447dc8f347c9188078e94: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260805 23:13:04.940748 8209 raft_consensus.cc:348] T 010a96d4aeb243df9570dd9adf632526 P 57ca592f72ed4d6f8fd9a07643f02fd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f3cc8ff1e664a859a891b80666307f1" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 37313 } } peers { permanent_uuid: "57ca592f72ed4d6f8fd9a07643f02fd5" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38153 } } peers { permanent_uuid: "9d7053e2a49447dc8f347c9188078e94" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 43069 } }
I20260805 23:13:04.941046 8209 raft_consensus.cc:374] T 010a96d4aeb243df9570dd9adf632526 P 57ca592f72ed4d6f8fd9a07643f02fd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:13:04.941108 8209 raft_consensus.cc:729] T 010a96d4aeb243df9570dd9adf632526 P 57ca592f72ed4d6f8fd9a07643f02fd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57ca592f72ed4d6f8fd9a07643f02fd5, State: Initialized, Role: FOLLOWER
I20260805 23:13:04.941679 8210 tablet_bootstrap.cc:492] T 010a96d4aeb243df9570dd9adf632526 P 3f3cc8ff1e664a859a891b80666307f1: No bootstrap required, opened a new log
I20260805 23:13:04.941658 8209 consensus_queue.cc:260] T 010a96d4aeb243df9570dd9adf632526 P 57ca592f72ed4d6f8fd9a07643f02fd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f3cc8ff1e664a859a891b80666307f1" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 37313 } } peers { permanent_uuid: "57ca592f72ed4d6f8fd9a07643f02fd5" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38153 } } peers { permanent_uuid: "9d7053e2a49447dc8f347c9188078e94" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 43069 } }
I20260805 23:13:04.941874 8210 ts_tablet_manager.cc:1397] T 010a96d4aeb243df9570dd9adf632526 P 3f3cc8ff1e664a859a891b80666307f1: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260805 23:13:04.942754 8209 ts_tablet_manager.cc:1428] T 010a96d4aeb243df9570dd9adf632526 P 57ca592f72ed4d6f8fd9a07643f02fd5: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260805 23:13:04.943162 8190 heartbeater.cc:503] Master 127.25.92.254:45961 was elected leader, sending a full tablet report...
I20260805 23:13:04.945766 8210 raft_consensus.cc:348] T 010a96d4aeb243df9570dd9adf632526 P 3f3cc8ff1e664a859a891b80666307f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f3cc8ff1e664a859a891b80666307f1" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 37313 } } peers { permanent_uuid: "57ca592f72ed4d6f8fd9a07643f02fd5" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38153 } } peers { permanent_uuid: "9d7053e2a49447dc8f347c9188078e94" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 43069 } }
I20260805 23:13:04.946070 8210 raft_consensus.cc:374] T 010a96d4aeb243df9570dd9adf632526 P 3f3cc8ff1e664a859a891b80666307f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:13:04.946121 8210 raft_consensus.cc:729] T 010a96d4aeb243df9570dd9adf632526 P 3f3cc8ff1e664a859a891b80666307f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f3cc8ff1e664a859a891b80666307f1, State: Initialized, Role: FOLLOWER
I20260805 23:13:04.946617 8210 consensus_queue.cc:260] T 010a96d4aeb243df9570dd9adf632526 P 3f3cc8ff1e664a859a891b80666307f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f3cc8ff1e664a859a891b80666307f1" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 37313 } } peers { permanent_uuid: "57ca592f72ed4d6f8fd9a07643f02fd5" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38153 } } peers { permanent_uuid: "9d7053e2a49447dc8f347c9188078e94" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 43069 } }
I20260805 23:13:04.947034 8211 raft_consensus.cc:348] T 010a96d4aeb243df9570dd9adf632526 P 9d7053e2a49447dc8f347c9188078e94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f3cc8ff1e664a859a891b80666307f1" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 37313 } } peers { permanent_uuid: "57ca592f72ed4d6f8fd9a07643f02fd5" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38153 } } peers { permanent_uuid: "9d7053e2a49447dc8f347c9188078e94" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 43069 } }
I20260805 23:13:04.947306 8211 raft_consensus.cc:374] T 010a96d4aeb243df9570dd9adf632526 P 9d7053e2a49447dc8f347c9188078e94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:13:04.947358 8211 raft_consensus.cc:729] T 010a96d4aeb243df9570dd9adf632526 P 9d7053e2a49447dc8f347c9188078e94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d7053e2a49447dc8f347c9188078e94, State: Initialized, Role: FOLLOWER
I20260805 23:13:04.947383 8050 heartbeater.cc:503] Master 127.25.92.254:45961 was elected leader, sending a full tablet report...
I20260805 23:13:04.947716 8210 ts_tablet_manager.cc:1428] T 010a96d4aeb243df9570dd9adf632526 P 3f3cc8ff1e664a859a891b80666307f1: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260805 23:13:04.947788 8211 consensus_queue.cc:260] T 010a96d4aeb243df9570dd9adf632526 P 9d7053e2a49447dc8f347c9188078e94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f3cc8ff1e664a859a891b80666307f1" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 37313 } } peers { permanent_uuid: "57ca592f72ed4d6f8fd9a07643f02fd5" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38153 } } peers { permanent_uuid: "9d7053e2a49447dc8f347c9188078e94" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 43069 } }
I20260805 23:13:04.948572 7912 heartbeater.cc:503] Master 127.25.92.254:45961 was elected leader, sending a full tablet report...
I20260805 23:13:04.948796 8211 ts_tablet_manager.cc:1428] T 010a96d4aeb243df9570dd9adf632526 P 9d7053e2a49447dc8f347c9188078e94: Time spent starting tablet: real 0.009s user 0.008s sys 0.000s
W20260805 23:13:05.105969 8192 tablet.cc:2369] T 010a96d4aeb243df9570dd9adf632526 P 57ca592f72ed4d6f8fd9a07643f02fd5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260805 23:13:05.117857 8051 tablet.cc:2369] T 010a96d4aeb243df9570dd9adf632526 P 3f3cc8ff1e664a859a891b80666307f1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260805 23:13:05.124668 7913 tablet.cc:2369] T 010a96d4aeb243df9570dd9adf632526 P 9d7053e2a49447dc8f347c9188078e94: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260805 23:13:05.213089 8217 raft_consensus.cc:482] T 010a96d4aeb243df9570dd9adf632526 P 9d7053e2a49447dc8f347c9188078e94 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 23:13:05.213377 8217 raft_consensus.cc:504] T 010a96d4aeb243df9570dd9adf632526 P 9d7053e2a49447dc8f347c9188078e94 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f3cc8ff1e664a859a891b80666307f1" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 37313 } } peers { permanent_uuid: "57ca592f72ed4d6f8fd9a07643f02fd5" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38153 } } peers { permanent_uuid: "9d7053e2a49447dc8f347c9188078e94" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 43069 } }
I20260805 23:13:05.214808 8217 leader_election.cc:288] T 010a96d4aeb243df9570dd9adf632526 P 9d7053e2a49447dc8f347c9188078e94 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f3cc8ff1e664a859a891b80666307f1 (127.25.92.194:37313), 57ca592f72ed4d6f8fd9a07643f02fd5 (127.25.92.195:38153)
I20260805 23:13:05.219419 8205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:05.215286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.193:49737 (local address 127.25.92.195:38153)
0805 23:13:05.215439 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0805 23:13:05.215444 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:13:05.215753 (+ 309us) server_negotiation.cc:407] Connection header received
0805 23:13:05.215901 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:13:05.215907 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:13:05.215969 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:13:05.216047 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:13:05.216462 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:13:05.217042 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:13:05.218345 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:13:05.218761 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:13:05.218899 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:13:05.218959 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0805 23:13:05.219195 (+ 236us) server_negotiation.cc:299] Negotiation successful
0805 23:13:05.219286 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260805 23:13:05.219444 8219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:05.215170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.194:37313 (local address 127.25.92.193:55677)
0805 23:13:05.215731 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0805 23:13:05.215749 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0805 23:13:05.215875 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:13:05.216200 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:13:05.216202 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:13:05.216216 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:13:05.216339 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:13:05.216344 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:13:05.217172 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:13:05.217176 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:13:05.218175 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:13:05.218186 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:13:05.218922 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:13:05.218925 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:13:05.219080 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:13:05.219099 (+ 19us) client_negotiation.cc:769] Sending connection context
0805 23:13:05.219203 (+ 104us) client_negotiation.cc:240] Negotiation successful
0805 23:13:05.219277 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":451,"mutex_wait_us":125,"thread_start_us":122,"threads_started":1}
I20260805 23:13:05.219446 8204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:05.215275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.193:55677 (local address 127.25.92.194:37313)
0805 23:13:05.215439 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0805 23:13:05.215444 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:13:05.215861 (+ 417us) server_negotiation.cc:407] Connection header received
0805 23:13:05.215992 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:13:05.215994 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:13:05.216084 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:13:05.216173 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:13:05.216494 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:13:05.217042 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:13:05.218312 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:13:05.218766 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:13:05.218914 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:13:05.218973 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0805 23:13:05.219240 (+ 267us) server_negotiation.cc:299] Negotiation successful
0805 23:13:05.219330 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260805 23:13:05.219458 8218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:05.215170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.195:38153 (local address 127.25.92.193:49737)
0805 23:13:05.215627 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0805 23:13:05.215640 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0805 23:13:05.215759 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:13:05.216085 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:13:05.216089 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:13:05.216110 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:13:05.216317 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:13:05.216325 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:13:05.217190 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:13:05.217194 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:13:05.218182 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:13:05.218191 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:13:05.218898 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:13:05.218902 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:13:05.219031 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:13:05.219077 (+ 46us) client_negotiation.cc:769] Sending connection context
0805 23:13:05.219146 (+ 69us) client_negotiation.cc:240] Negotiation successful
0805 23:13:05.219255 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":389,"mutex_wait_us":40,"thread_start_us":150,"threads_started":1}
I20260805 23:13:05.220336 8002 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "010a96d4aeb243df9570dd9adf632526" candidate_uuid: "9d7053e2a49447dc8f347c9188078e94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f3cc8ff1e664a859a891b80666307f1" is_pre_election: true
I20260805 23:13:05.220486 8142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "010a96d4aeb243df9570dd9adf632526" candidate_uuid: "9d7053e2a49447dc8f347c9188078e94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57ca592f72ed4d6f8fd9a07643f02fd5" is_pre_election: true
I20260805 23:13:05.220731 8002 raft_consensus.cc:2454] T 010a96d4aeb243df9570dd9adf632526 P 3f3cc8ff1e664a859a891b80666307f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d7053e2a49447dc8f347c9188078e94 in term 0.
I20260805 23:13:05.220897 8142 raft_consensus.cc:2454] T 010a96d4aeb243df9570dd9adf632526 P 57ca592f72ed4d6f8fd9a07643f02fd5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d7053e2a49447dc8f347c9188078e94 in term 0.
I20260805 23:13:05.221377 7797 leader_election.cc:302] T 010a96d4aeb243df9570dd9adf632526 P 9d7053e2a49447dc8f347c9188078e94 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f3cc8ff1e664a859a891b80666307f1, 9d7053e2a49447dc8f347c9188078e94; no voters:
I20260805 23:13:05.221760 8217 raft_consensus.cc:2781] T 010a96d4aeb243df9570dd9adf632526 P 9d7053e2a49447dc8f347c9188078e94 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260805 23:13:05.221866 8217 raft_consensus.cc:482] T 010a96d4aeb243df9570dd9adf632526 P 9d7053e2a49447dc8f347c9188078e94 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260805 23:13:05.221925 8217 raft_consensus.cc:3037] T 010a96d4aeb243df9570dd9adf632526 P 9d7053e2a49447dc8f347c9188078e94 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:13:05.223088 8217 raft_consensus.cc:504] T 010a96d4aeb243df9570dd9adf632526 P 9d7053e2a49447dc8f347c9188078e94 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f3cc8ff1e664a859a891b80666307f1" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 37313 } } peers { permanent_uuid: "57ca592f72ed4d6f8fd9a07643f02fd5" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38153 } } peers { permanent_uuid: "9d7053e2a49447dc8f347c9188078e94" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 43069 } }
I20260805 23:13:05.223562 8217 leader_election.cc:288] T 010a96d4aeb243df9570dd9adf632526 P 9d7053e2a49447dc8f347c9188078e94 [CANDIDATE]: Term 1 election: Requested vote from peers 3f3cc8ff1e664a859a891b80666307f1 (127.25.92.194:37313), 57ca592f72ed4d6f8fd9a07643f02fd5 (127.25.92.195:38153)
I20260805 23:13:05.224036 8142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "010a96d4aeb243df9570dd9adf632526" candidate_uuid: "9d7053e2a49447dc8f347c9188078e94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57ca592f72ed4d6f8fd9a07643f02fd5"
I20260805 23:13:05.224102 8002 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "010a96d4aeb243df9570dd9adf632526" candidate_uuid: "9d7053e2a49447dc8f347c9188078e94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f3cc8ff1e664a859a891b80666307f1"
I20260805 23:13:05.224218 8142 raft_consensus.cc:3037] T 010a96d4aeb243df9570dd9adf632526 P 57ca592f72ed4d6f8fd9a07643f02fd5 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:13:05.224272 8002 raft_consensus.cc:3037] T 010a96d4aeb243df9570dd9adf632526 P 3f3cc8ff1e664a859a891b80666307f1 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:13:05.225630 8002 raft_consensus.cc:2454] T 010a96d4aeb243df9570dd9adf632526 P 3f3cc8ff1e664a859a891b80666307f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d7053e2a49447dc8f347c9188078e94 in term 1.
I20260805 23:13:05.225723 8142 raft_consensus.cc:2454] T 010a96d4aeb243df9570dd9adf632526 P 57ca592f72ed4d6f8fd9a07643f02fd5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d7053e2a49447dc8f347c9188078e94 in term 1.
I20260805 23:13:05.226025 7797 leader_election.cc:302] T 010a96d4aeb243df9570dd9adf632526 P 9d7053e2a49447dc8f347c9188078e94 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f3cc8ff1e664a859a891b80666307f1, 9d7053e2a49447dc8f347c9188078e94; no voters:
I20260805 23:13:05.226258 8217 raft_consensus.cc:2781] T 010a96d4aeb243df9570dd9adf632526 P 9d7053e2a49447dc8f347c9188078e94 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:13:05.226514 8217 raft_consensus.cc:686] T 010a96d4aeb243df9570dd9adf632526 P 9d7053e2a49447dc8f347c9188078e94 [term 1 LEADER]: Becoming Leader. State: Replica: 9d7053e2a49447dc8f347c9188078e94, State: Running, Role: LEADER
I20260805 23:13:05.226897 8217 consensus_queue.cc:237] T 010a96d4aeb243df9570dd9adf632526 P 9d7053e2a49447dc8f347c9188078e94 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f3cc8ff1e664a859a891b80666307f1" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 37313 } } peers { permanent_uuid: "57ca592f72ed4d6f8fd9a07643f02fd5" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38153 } } peers { permanent_uuid: "9d7053e2a49447dc8f347c9188078e94" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 43069 } }
I20260805 23:13:05.230826 7715 catalog_manager.cc:5495] T 010a96d4aeb243df9570dd9adf632526 P 9d7053e2a49447dc8f347c9188078e94 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9d7053e2a49447dc8f347c9188078e94 (127.25.92.193). New cstate: current_term: 1 leader_uuid: "9d7053e2a49447dc8f347c9188078e94" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f3cc8ff1e664a859a891b80666307f1" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 37313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57ca592f72ed4d6f8fd9a07643f02fd5" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d7053e2a49447dc8f347c9188078e94" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 43069 } health_report { overall_health: HEALTHY } } }
I20260805 23:13:05.285349 8207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:05.281257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43058 (local address 127.25.92.193:43069)
0805 23:13:05.281372 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0805 23:13:05.281376 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:13:05.281392 (+ 16us) server_negotiation.cc:407] Connection header received
0805 23:13:05.281471 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:13:05.281473 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:13:05.281592 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:13:05.281682 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 23:13:05.281978 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:13:05.282615 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:13:05.283495 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:13:05.283780 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:13:05.283870 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:13:05.284277 (+ 407us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 23:13:05.284922 (+ 645us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 23:13:05.285050 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0805 23:13:05.285120 (+ 70us) server_negotiation.cc:299] Negotiation successful
0805 23:13:05.285180 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260805 23:13:05.291721 8142 raft_consensus.cc:1261] T 010a96d4aeb243df9570dd9adf632526 P 57ca592f72ed4d6f8fd9a07643f02fd5 [term 1 FOLLOWER]: Refusing update from remote peer 9d7053e2a49447dc8f347c9188078e94: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:13:05.291755 8002 raft_consensus.cc:1261] T 010a96d4aeb243df9570dd9adf632526 P 3f3cc8ff1e664a859a891b80666307f1 [term 1 FOLLOWER]: Refusing update from remote peer 9d7053e2a49447dc8f347c9188078e94: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:13:05.292531 8217 consensus_queue.cc:1035] T 010a96d4aeb243df9570dd9adf632526 P 9d7053e2a49447dc8f347c9188078e94 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f3cc8ff1e664a859a891b80666307f1" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 37313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:13:05.292788 8220 consensus_queue.cc:1035] T 010a96d4aeb243df9570dd9adf632526 P 9d7053e2a49447dc8f347c9188078e94 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57ca592f72ed4d6f8fd9a07643f02fd5" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:13:05.303443 8225 mvcc.cc:205] Tried to move back new op lower bound from 7315339613346463744 to 7315339613092212736. Current Snapshot: MvccSnapshot[applied={T|T < 7315339613346463744}]
I20260805 23:13:05.306455 8224 mvcc.cc:205] Tried to move back new op lower bound from 7315339613346463744 to 7315339613092212736. Current Snapshot: MvccSnapshot[applied={T|T < 7315339613346463744 or (T in {7315339613346463744})}]
I20260805 23:13:05.310585 7715 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48550:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:13:05.310787 7715 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48550:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:13:05.313550 7715 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cea136ae38a74d03b8dfdfc98787079d: Sending DeleteTablet for 3 replicas of tablet 010a96d4aeb243df9570dd9adf632526
I20260805 23:13:05.314483 7982 tablet_service.cc:1514] Processing DeleteTablet for tablet 010a96d4aeb243df9570dd9adf632526 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:13:05 UTC) from {username='kudu', principal='kudu/127.25.92.254@KRBTEST.COM'} at 127.0.0.1:54820
I20260805 23:13:05.314798 7843 tablet_service.cc:1514] Processing DeleteTablet for tablet 010a96d4aeb243df9570dd9adf632526 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:13:05 UTC) from {username='kudu', principal='kudu/127.25.92.254@KRBTEST.COM'} at 127.0.0.1:43056
I20260805 23:13:05.315042 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 7779
I20260805 23:13:05.315109 8122 tablet_service.cc:1514] Processing DeleteTablet for tablet 010a96d4aeb243df9570dd9adf632526 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:13:05 UTC) from {username='kudu', principal='kudu/127.25.92.254@KRBTEST.COM'} at 127.0.0.1:59332
I20260805 23:13:05.316426 8234 tablet_replica.cc:330] T 010a96d4aeb243df9570dd9adf632526 P 57ca592f72ed4d6f8fd9a07643f02fd5: stopping tablet replica
I20260805 23:13:05.316713 8234 raft_consensus.cc:2229] T 010a96d4aeb243df9570dd9adf632526 P 57ca592f72ed4d6f8fd9a07643f02fd5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 23:13:05.316992 8234 raft_consensus.cc:2258] T 010a96d4aeb243df9570dd9adf632526 P 57ca592f72ed4d6f8fd9a07643f02fd5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:13:05.318617 8234 ts_tablet_manager.cc:1905] T 010a96d4aeb243df9570dd9adf632526 P 57ca592f72ed4d6f8fd9a07643f02fd5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:13:05.320839 8234 ts_tablet_manager.cc:1918] T 010a96d4aeb243df9570dd9adf632526 P 57ca592f72ed4d6f8fd9a07643f02fd5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 23:13:05.320922 8234 log.cc:1196] T 010a96d4aeb243df9570dd9adf632526 P 57ca592f72ed4d6f8fd9a07643f02fd5: Deleting WAL directory at /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-2/wal/wals/010a96d4aeb243df9570dd9adf632526
I20260805 23:13:05.321168 8234 ts_tablet_manager.cc:1939] T 010a96d4aeb243df9570dd9adf632526 P 57ca592f72ed4d6f8fd9a07643f02fd5: Deleting consensus metadata
I20260805 23:13:05.321970 8232 tablet_replica.cc:330] T 010a96d4aeb243df9570dd9adf632526 P 3f3cc8ff1e664a859a891b80666307f1: stopping tablet replica
I20260805 23:13:05.322376 8232 raft_consensus.cc:2229] T 010a96d4aeb243df9570dd9adf632526 P 3f3cc8ff1e664a859a891b80666307f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 23:13:05.322937 8232 raft_consensus.cc:2258] T 010a96d4aeb243df9570dd9adf632526 P 3f3cc8ff1e664a859a891b80666307f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:13:05.322918 7699 catalog_manager.cc:4841] TS 57ca592f72ed4d6f8fd9a07643f02fd5 (127.25.92.195:38153): tablet 010a96d4aeb243df9570dd9adf632526 (table test-table [id=a4d0cd17e8314dcc866a577685d67a66]) successfully deleted
W20260805 23:13:05.323487 7701 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.25.92.193:43069 (error 108)
I20260805 23:13:05.323748 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 7917
I20260805 23:13:05.327245 8203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:05.326915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.193:43069 (local address 127.0.0.1:43066)
0805 23:13:05.327071 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0805 23:13:05.327161 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.193:43069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260805 23:13:05.327498 7701 catalog_manager.cc:4584] TS 9d7053e2a49447dc8f347c9188078e94 (127.25.92.193:43069): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 010a96d4aeb243df9570dd9adf632526: Network error: Client connection negotiation failed: client connection to 127.25.92.193:43069: connect: Connection refused (error 111)
I20260805 23:13:05.331418 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 8055
I20260805 23:13:05.332790 8203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:05.331576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.194:37313 (local address 127.0.0.1:54828)
0805 23:13:05.332548 (+ 972us) negotiation.cc:107] Waiting for socket to connect
0805 23:13:05.332622 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.194:37313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":856}
I20260805 23:13:05.337682 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 7681
I20260805 23:13:05.345214 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.254:45961
--webserver_interface=127.25.92.254
--webserver_port=46373
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab
--principal=kudu/127.25.92.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:41983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.92.254:45961
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:13:05.463299 8236 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:13:05.463657 8236 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:13:05.463738 8236 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:13:05.468037 8236 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:13:05.468169 8236 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:13:05.468210 8236 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:13:05.468416 8236 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:13:05.468598 8236 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:13:05.475224 8236 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:41983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.25.92.254:45961
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab
--principal=kudu/127.25.92.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.254:45961
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.92.254
--webserver_port=46373
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8236
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:13:05.476532 8236 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:13:05.477797 8236 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:13:05.493865 8245 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:13:05.493853 8243 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:13:05.493937 8236 server_base.cc:1034] running on GCE node
W20260805 23:13:05.493846 8242 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:13:05.494870 8236 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:13:05.496079 8236 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:13:05.497268 8236 hybrid_clock.cc:648] HybridClock initialized: now 1785971585497235 us; error 51 us; skew 500 ppm
Aug 05 23:13:05 dist-test-slave-ss4n krb5kdc[7668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971585, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:13:05.500967 8236 init.cc:377] Logged in from keytab as kudu/127.25.92.254@KRBTEST.COM (short username kudu)
I20260805 23:13:05.502282 8236 webserver.cc:460] Webserver started at http://127.25.92.254:46373/ using document root <none> and password file <none>
I20260805 23:13:05.502980 8236 fs_manager.cc:359] Metadata directory not provided
I20260805 23:13:05.503072 8236 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:13:05.507879 8236 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260805 23:13:05.510551 8252 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:13:05.512010 8236 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:13:05.512197 8236 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/master-0/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/master-0/wal
uuid: "cea136ae38a74d03b8dfdfc98787079d"
format_stamp: "Formatted at 2026-08-05 23:13:03 on dist-test-slave-ss4n"
server_key: "f5eff3ff5f4038c6d9d0ec33d02b3020"
server_key_iv: "4787f4b860dd10e3df762e1ae1ff1bad"
server_key_version: "encryptionkey@0"
I20260805 23:13:05.512938 8236 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:13:05.530931 8236 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:13:05.534690 8236 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:13:05.534932 8236 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:13:05.542850 8236 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.254:45961
I20260805 23:13:05.543188 8307 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.254:45961 every 8 connection(s)
I20260805 23:13:05.544113 8236 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
I20260805 23:13:05.548722 8308 sys_catalog.cc:263] Verifying existing consensus state
I20260805 23:13:05.550591 8308 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cea136ae38a74d03b8dfdfc98787079d: Bootstrap starting.
I20260805 23:13:05.553323 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 8236
I20260805 23:13:05.553907 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.92.193:43069
--local_ip_for_outbound_sockets=127.25.92.193
--tserver_master_addrs=127.25.92.254:45961
--webserver_port=36179
--webserver_interface=127.25.92.193
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.193.keytab
--principal=kudu/127.25.92.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:41983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260805 23:13:05.568902 8308 log.cc:824] T 00000000000000000000000000000000 P cea136ae38a74d03b8dfdfc98787079d: Log is configured to *not* fsync() on all Append() calls
I20260805 23:13:05.578162 8308 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cea136ae38a74d03b8dfdfc98787079d: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260805 23:13:05.578604 8308 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cea136ae38a74d03b8dfdfc98787079d: Bootstrap complete.
I20260805 23:13:05.582995 8308 raft_consensus.cc:348] T 00000000000000000000000000000000 P cea136ae38a74d03b8dfdfc98787079d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cea136ae38a74d03b8dfdfc98787079d" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 45961 } }
I20260805 23:13:05.583439 8308 raft_consensus.cc:729] T 00000000000000000000000000000000 P cea136ae38a74d03b8dfdfc98787079d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cea136ae38a74d03b8dfdfc98787079d, State: Initialized, Role: FOLLOWER
I20260805 23:13:05.584053 8308 consensus_queue.cc:260] T 00000000000000000000000000000000 P cea136ae38a74d03b8dfdfc98787079d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cea136ae38a74d03b8dfdfc98787079d" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 45961 } }
I20260805 23:13:05.584224 8308 raft_consensus.cc:388] T 00000000000000000000000000000000 P cea136ae38a74d03b8dfdfc98787079d [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:13:05.584321 8308 raft_consensus.cc:482] T 00000000000000000000000000000000 P cea136ae38a74d03b8dfdfc98787079d [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:13:05.584467 8308 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cea136ae38a74d03b8dfdfc98787079d [term 1 FOLLOWER]: Advancing to term 2
I20260805 23:13:05.586583 8308 raft_consensus.cc:504] T 00000000000000000000000000000000 P cea136ae38a74d03b8dfdfc98787079d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cea136ae38a74d03b8dfdfc98787079d" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 45961 } }
I20260805 23:13:05.586925 8308 leader_election.cc:302] T 00000000000000000000000000000000 P cea136ae38a74d03b8dfdfc98787079d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cea136ae38a74d03b8dfdfc98787079d; no voters:
I20260805 23:13:05.587286 8308 leader_election.cc:288] T 00000000000000000000000000000000 P cea136ae38a74d03b8dfdfc98787079d [CANDIDATE]: Term 2 election: Requested vote from peers
I20260805 23:13:05.587386 8312 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cea136ae38a74d03b8dfdfc98787079d [term 2 FOLLOWER]: Leader election won for term 2
I20260805 23:13:05.587615 8312 raft_consensus.cc:686] T 00000000000000000000000000000000 P cea136ae38a74d03b8dfdfc98787079d [term 2 LEADER]: Becoming Leader. State: Replica: cea136ae38a74d03b8dfdfc98787079d, State: Running, Role: LEADER
I20260805 23:13:05.587913 8312 consensus_queue.cc:237] T 00000000000000000000000000000000 P cea136ae38a74d03b8dfdfc98787079d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cea136ae38a74d03b8dfdfc98787079d" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 45961 } }
I20260805 23:13:05.588600 8308 sys_catalog.cc:564] T 00000000000000000000000000000000 P cea136ae38a74d03b8dfdfc98787079d [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:13:05.588896 8312 sys_catalog.cc:455] T 00000000000000000000000000000000 P cea136ae38a74d03b8dfdfc98787079d [sys.catalog]: SysCatalogTable state changed. Reason: New leader cea136ae38a74d03b8dfdfc98787079d. Latest consensus state: current_term: 2 leader_uuid: "cea136ae38a74d03b8dfdfc98787079d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cea136ae38a74d03b8dfdfc98787079d" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 45961 } } }
I20260805 23:13:05.589020 8312 sys_catalog.cc:458] T 00000000000000000000000000000000 P cea136ae38a74d03b8dfdfc98787079d [sys.catalog]: This master's current role is: LEADER
I20260805 23:13:05.589381 8316 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:13:05.589542 8313 sys_catalog.cc:455] T 00000000000000000000000000000000 P cea136ae38a74d03b8dfdfc98787079d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "cea136ae38a74d03b8dfdfc98787079d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cea136ae38a74d03b8dfdfc98787079d" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 45961 } } }
I20260805 23:13:05.589747 8313 sys_catalog.cc:458] T 00000000000000000000000000000000 P cea136ae38a74d03b8dfdfc98787079d [sys.catalog]: This master's current role is: LEADER
I20260805 23:13:05.594496 8316 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:13:05.595219 8316 catalog_manager.cc:1259] Loaded cluster ID: 31a5fac73c5b4a22a381a0c99d3ea80a
I20260805 23:13:05.595309 8316 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:13:05.596439 8316 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260805 23:13:05.599356 8316 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260805 23:13:05.604355 8316 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:13:05.605211 8316 catalog_manager.cc:5879] T 00000000000000000000000000000000 P cea136ae38a74d03b8dfdfc98787079d: Loaded TSK: 0
I20260805 23:13:05.606369 8316 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260805 23:13:05.695608 8310 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:13:05.695972 8310 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:13:05.696085 8310 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:13:05.700474 8310 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:13:05.700598 8310 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:13:05.700698 8310 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.92.193
I20260805 23:13:05.706636 8310 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:41983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.193.keytab
--principal=kudu/127.25.92.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.92.193:43069
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.92.193
--webserver_port=36179
--tserver_master_addrs=127.25.92.254:45961
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8310
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.92.193
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:13:05.707818 8310 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:13:05.708982 8310 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:13:05.720011 8336 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:13:05.719980 8335 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:13:05.719980 8338 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:13:05.721060 8310 server_base.cc:1034] running on GCE node
I20260805 23:13:05.721665 8310 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:13:05.722411 8310 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:13:05.723637 8310 hybrid_clock.cc:648] HybridClock initialized: now 1785971585723616 us; error 44 us; skew 500 ppm
Aug 05 23:13:05 dist-test-slave-ss4n krb5kdc[7668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971585, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:13:05.727248 8310 init.cc:377] Logged in from keytab as kudu/127.25.92.193@KRBTEST.COM (short username kudu)
I20260805 23:13:05.728468 8310 webserver.cc:460] Webserver started at http://127.25.92.193:36179/ using document root <none> and password file <none>
I20260805 23:13:05.729102 8310 fs_manager.cc:359] Metadata directory not provided
I20260805 23:13:05.729231 8310 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:13:05.734279 8310 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:13:05.737097 8345 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:13:05.738790 8310 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:13:05.738988 8310 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-0/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-0/wal
uuid: "9d7053e2a49447dc8f347c9188078e94"
format_stamp: "Formatted at 2026-08-05 23:13:04 on dist-test-slave-ss4n"
server_key: "d5e2b7c396a15bd3a8b77add6d817f32"
server_key_iv: "f0c4bb51274444f28601105a60511d8d"
server_key_version: "encryptionkey@0"
I20260805 23:13:05.739667 8310 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:13:05.755318 8310 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:13:05.759145 8310 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:13:05.759400 8310 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:13:05.760062 8310 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:13:05.761425 8352 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260805 23:13:05.766144 8310 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260805 23:13:05.766222 8310 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260805 23:13:05.766306 8310 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260805 23:13:05.768535 8310 ts_tablet_manager.cc:610] Registered 1 tablets
I20260805 23:13:05.768613 8310 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260805 23:13:05.769001 8352 tablet_bootstrap.cc:492] T 010a96d4aeb243df9570dd9adf632526 P 9d7053e2a49447dc8f347c9188078e94: Bootstrap starting.
I20260805 23:13:05.779887 8310 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.193:43069
I20260805 23:13:05.780860 8310 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-0/data/info.pb
I20260805 23:13:05.781622 8464 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.193:43069 every 8 connection(s)
I20260805 23:13:05.782485 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 8310
I20260805 23:13:05.783066 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.92.194:37313
--local_ip_for_outbound_sockets=127.25.92.194
--tserver_master_addrs=127.25.92.254:45961
--webserver_port=44697
--webserver_interface=127.25.92.194
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.194.keytab
--principal=kudu/127.25.92.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:41983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260805 23:13:05.789502 8352 log.cc:824] T 010a96d4aeb243df9570dd9adf632526 P 9d7053e2a49447dc8f347c9188078e94: Log is configured to *not* fsync() on all Append() calls
Aug 05 23:13:05 dist-test-slave-ss4n krb5kdc[7668](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971585, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.193@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM
I20260805 23:13:05.798026 8352 tablet_bootstrap.cc:492] T 010a96d4aeb243df9570dd9adf632526 P 9d7053e2a49447dc8f347c9188078e94: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260805 23:13:05.798517 8352 tablet_bootstrap.cc:492] T 010a96d4aeb243df9570dd9adf632526 P 9d7053e2a49447dc8f347c9188078e94: Bootstrap complete.
I20260805 23:13:05.799803 8352 ts_tablet_manager.cc:1397] T 010a96d4aeb243df9570dd9adf632526 P 9d7053e2a49447dc8f347c9188078e94: Time spent bootstrapping tablet: real 0.031s user 0.020s sys 0.007s
I20260805 23:13:05.802920 8352 raft_consensus.cc:348] T 010a96d4aeb243df9570dd9adf632526 P 9d7053e2a49447dc8f347c9188078e94 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f3cc8ff1e664a859a891b80666307f1" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 37313 } } peers { permanent_uuid: "57ca592f72ed4d6f8fd9a07643f02fd5" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38153 } } peers { permanent_uuid: "9d7053e2a49447dc8f347c9188078e94" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 43069 } }
I20260805 23:13:05.803495 8352 raft_consensus.cc:729] T 010a96d4aeb243df9570dd9adf632526 P 9d7053e2a49447dc8f347c9188078e94 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d7053e2a49447dc8f347c9188078e94, State: Initialized, Role: FOLLOWER
I20260805 23:13:05.804049 8352 consensus_queue.cc:260] T 010a96d4aeb243df9570dd9adf632526 P 9d7053e2a49447dc8f347c9188078e94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f3cc8ff1e664a859a891b80666307f1" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 37313 } } peers { permanent_uuid: "57ca592f72ed4d6f8fd9a07643f02fd5" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38153 } } peers { permanent_uuid: "9d7053e2a49447dc8f347c9188078e94" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 43069 } }
I20260805 23:13:05.804391 8468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:05.783272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:45961 (local address 127.25.92.193:40059)
0805 23:13:05.785915 (+ 2643us) negotiation.cc:107] Waiting for socket to connect
0805 23:13:05.785976 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0805 23:13:05.787351 (+ 1375us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:13:05.788803 (+ 1452us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:13:05.788813 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:13:05.789298 (+ 485us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:13:05.789956 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:13:05.789982 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:13:05.791726 (+ 1744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:13:05.791732 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:13:05.793386 (+ 1654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:13:05.793404 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:13:05.794510 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:13:05.794517 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:13:05.794777 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:13:05.795575 (+ 798us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:13:05.795602 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:13:05.797665 (+ 2063us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:13:05.800455 (+ 2790us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:13:05.800462 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:13:05.800476 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:13:05.800844 (+ 368us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:13:05.801271 (+ 427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:13:05.801274 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:13:05.801276 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:13:05.801456 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:13:05.802312 (+ 856us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:13:05.802321 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:13:05.802693 (+ 372us) client_negotiation.cc:769] Sending connection context
0805 23:13:05.802924 (+ 231us) client_negotiation.cc:240] Negotiation successful
0805 23:13:05.803295 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2378,"thread_start_us":155,"threads_started":1}
I20260805 23:13:05.804854 8467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:05.783301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.193:40059 (local address 127.25.92.254:45961)
0805 23:13:05.784982 (+ 1681us) server_negotiation.cc:206] Beginning negotiation
0805 23:13:05.784998 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0805 23:13:05.786115 (+ 1117us) server_negotiation.cc:407] Connection header received
0805 23:13:05.787868 (+ 1753us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:13:05.787886 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:13:05.788329 (+ 443us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:13:05.788748 (+ 419us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:13:05.790204 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:13:05.791528 (+ 1324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:13:05.793598 (+ 2070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:13:05.794329 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:13:05.794504 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:13:05.797918 (+ 3414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:13:05.797926 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:13:05.797943 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:13:05.797981 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:13:05.800301 (+ 2320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:13:05.800990 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:13:05.800996 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:13:05.801002 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:13:05.801164 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:13:05.801613 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:13:05.801617 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:13:05.801619 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:13:05.802157 (+ 538us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:13:05.802321 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0805 23:13:05.803109 (+ 788us) server_negotiation.cc:299] Negotiation successful
0805 23:13:05.803614 (+ 505us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1514,"thread_start_us":120,"threads_started":1}
I20260805 23:13:05.805190 8352 ts_tablet_manager.cc:1428] T 010a96d4aeb243df9570dd9adf632526 P 9d7053e2a49447dc8f347c9188078e94: Time spent starting tablet: real 0.005s user 0.006s sys 0.001s
I20260805 23:13:05.807492 8465 heartbeater.cc:347] Connected to a master server at 127.25.92.254:45961
I20260805 23:13:05.807765 8465 heartbeater.cc:464] Registering TS with master...
I20260805 23:13:05.808287 8465 heartbeater.cc:511] Master 127.25.92.254:45961 requested a full tablet report, sending...
I20260805 23:13:05.810847 8270 ts_manager.cc:194] Registered new tserver with Master: 9d7053e2a49447dc8f347c9188078e94 (127.25.92.193:43069)
I20260805 23:13:05.814500 8270 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.92.193@KRBTEST.COM'} at 127.25.92.193:40059
I20260805 23:13:05.817090 8465 heartbeater.cc:503] Master 127.25.92.254:45961 was elected leader, sending a full tablet report...
I20260805 23:13:05.822551 8473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:05.814748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43076 (local address 127.25.92.193:43069)
0805 23:13:05.817756 (+ 3008us) server_negotiation.cc:206] Beginning negotiation
0805 23:13:05.817769 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0805 23:13:05.817792 (+ 23us) server_negotiation.cc:407] Connection header received
0805 23:13:05.817863 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:13:05.817869 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:13:05.818055 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:13:05.818610 (+ 555us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:13:05.818747 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:13:05.819536 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:13:05.821212 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:13:05.821849 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:13:05.821945 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:13:05.822050 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0805 23:13:05.822356 (+ 306us) server_negotiation.cc:299] Negotiation successful
0805 23:13:05.822434 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2900,"thread_start_us":125,"threads_started":1}
I20260805 23:13:05.822635 8472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:05.813875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.193:43069 (local address 127.0.0.1:43076)
0805 23:13:05.815371 (+ 1496us) negotiation.cc:107] Waiting for socket to connect
0805 23:13:05.815419 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0805 23:13:05.815551 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:13:05.818257 (+ 2706us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:13:05.818267 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:13:05.818295 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:13:05.818436 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:13:05.818445 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:13:05.819686 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:13:05.819689 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:13:05.821048 (+ 1359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:13:05.821061 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:13:05.821996 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:13:05.822002 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:13:05.822131 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:13:05.822188 (+ 57us) client_negotiation.cc:769] Sending connection context
0805 23:13:05.822324 (+ 136us) client_negotiation.cc:240] Negotiation successful
0805 23:13:05.822470 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1312,"thread_start_us":129,"threads_started":1}
I20260805 23:13:05.823812 8395 tablet_service.cc:1514] Processing DeleteTablet for tablet 010a96d4aeb243df9570dd9adf632526 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:13:05 UTC) from {username='kudu', principal='kudu/127.25.92.254@KRBTEST.COM'} at 127.0.0.1:43076
I20260805 23:13:05.824851 8474 tablet_replica.cc:330] stopping tablet replica
I20260805 23:13:05.825170 8474 raft_consensus.cc:2229] T 010a96d4aeb243df9570dd9adf632526 P 9d7053e2a49447dc8f347c9188078e94 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 23:13:05.825430 8474 raft_consensus.cc:2258] T 010a96d4aeb243df9570dd9adf632526 P 9d7053e2a49447dc8f347c9188078e94 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:13:05.826480 8474 ts_tablet_manager.cc:1905] T 010a96d4aeb243df9570dd9adf632526 P 9d7053e2a49447dc8f347c9188078e94: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:13:05.830268 8474 ts_tablet_manager.cc:1918] T 010a96d4aeb243df9570dd9adf632526 P 9d7053e2a49447dc8f347c9188078e94: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 23:13:05.830415 8474 log.cc:1196] T 010a96d4aeb243df9570dd9adf632526 P 9d7053e2a49447dc8f347c9188078e94: Deleting WAL directory at /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-0/wal/wals/010a96d4aeb243df9570dd9adf632526
I20260805 23:13:05.830708 8474 ts_tablet_manager.cc:1939] T 010a96d4aeb243df9570dd9adf632526 P 9d7053e2a49447dc8f347c9188078e94: Deleting consensus metadata
I20260805 23:13:05.832557 8256 catalog_manager.cc:4841] TS 9d7053e2a49447dc8f347c9188078e94 (127.25.92.193:43069): tablet 010a96d4aeb243df9570dd9adf632526 (table test-table [id=a4d0cd17e8314dcc866a577685d67a66]) successfully deleted
W20260805 23:13:05.909729 8469 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:13:05.910054 8469 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:13:05.910101 8469 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:13:05.914769 8469 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:13:05.915007 8469 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:13:05.915164 8469 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.92.194
I20260805 23:13:05.923895 8469 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:41983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.194.keytab
--principal=kudu/127.25.92.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.92.194:37313
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.92.194
--webserver_port=44697
--tserver_master_addrs=127.25.92.254:45961
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8469
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.92.194
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:13:05.925156 8469 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:13:05.926260 8469 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:13:05.936477 8479 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:13:05.936477 8482 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:13:05.936539 8480 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:13:05.936816 8469 server_base.cc:1034] running on GCE node
I20260805 23:13:05.937309 8469 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:13:05.938023 8469 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:13:05.939217 8469 hybrid_clock.cc:648] HybridClock initialized: now 1785971585939185 us; error 43 us; skew 500 ppm
Aug 05 23:13:05 dist-test-slave-ss4n krb5kdc[7668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971585, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:13:05.942395 8469 init.cc:377] Logged in from keytab as kudu/127.25.92.194@KRBTEST.COM (short username kudu)
I20260805 23:13:05.943365 8469 webserver.cc:460] Webserver started at http://127.25.92.194:44697/ using document root <none> and password file <none>
I20260805 23:13:05.944068 8469 fs_manager.cc:359] Metadata directory not provided
I20260805 23:13:05.944150 8469 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:13:05.948346 8469 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260805 23:13:05.950796 8489 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:13:05.952333 8469 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:13:05.952482 8469 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-1/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-1/wal
uuid: "3f3cc8ff1e664a859a891b80666307f1"
format_stamp: "Formatted at 2026-08-05 23:13:04 on dist-test-slave-ss4n"
server_key: "0831a59f62202ce72c947f5a4bdb6bc4"
server_key_iv: "803573aa0d560547b97631878b70eb79"
server_key_version: "encryptionkey@0"
I20260805 23:13:05.953063 8469 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:13:05.971465 8469 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:13:05.975063 8469 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:13:05.975232 8469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:13:05.975786 8469 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:13:05.976892 8496 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260805 23:13:05.981812 8469 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260805 23:13:05.981930 8469 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20260805 23:13:05.982002 8469 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260805 23:13:05.983814 8469 ts_tablet_manager.cc:610] Registered 1 tablets
I20260805 23:13:05.983871 8469 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260805 23:13:05.984323 8496 tablet_bootstrap.cc:492] T 010a96d4aeb243df9570dd9adf632526 P 3f3cc8ff1e664a859a891b80666307f1: Bootstrap starting.
I20260805 23:13:05.995850 8469 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.194:37313
I20260805 23:13:05.995921 8607 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.194:37313 every 8 connection(s)
I20260805 23:13:05.996884 8469 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-1/data/info.pb
I20260805 23:13:06.002466 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 8469
I20260805 23:13:06.003006 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.92.195:38153
--local_ip_for_outbound_sockets=127.25.92.195
--tserver_master_addrs=127.25.92.254:45961
--webserver_port=44611
--webserver_interface=127.25.92.195
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.195.keytab
--principal=kudu/127.25.92.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:41983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260805 23:13:06.006297 8496 log.cc:824] T 010a96d4aeb243df9570dd9adf632526 P 3f3cc8ff1e664a859a891b80666307f1: Log is configured to *not* fsync() on all Append() calls
Aug 05 23:13:06 dist-test-slave-ss4n krb5kdc[7668](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971585, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.194@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM
I20260805 23:13:06.013268 8467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:05.999287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.194:48271 (local address 127.25.92.254:45961)
0805 23:13:05.999476 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0805 23:13:05.999482 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:13:06.000138 (+ 656us) server_negotiation.cc:407] Connection header received
0805 23:13:06.001147 (+ 1009us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:13:06.001151 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:13:06.001218 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:13:06.001338 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:13:06.002816 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:13:06.003563 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:13:06.004897 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:13:06.005337 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:13:06.005436 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:13:06.008844 (+ 3408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:13:06.008848 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:13:06.008850 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:13:06.008880 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:13:06.010829 (+ 1949us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:13:06.011441 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:13:06.011447 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:13:06.011449 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:13:06.011540 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:13:06.011909 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:13:06.011913 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:13:06.011915 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:13:06.012116 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:13:06.012232 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0805 23:13:06.012767 (+ 535us) server_negotiation.cc:299] Negotiation successful
0805 23:13:06.013058 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260805 23:13:06.014289 8610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:05.999409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:45961 (local address 127.25.92.194:48271)
0805 23:13:05.999941 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0805 23:13:06.000001 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0805 23:13:06.000906 (+ 905us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:13:06.001595 (+ 689us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:13:06.001604 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:13:06.002025 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:13:06.002591 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:13:06.002617 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:13:06.003746 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:13:06.003752 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:13:06.004696 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:13:06.004707 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:13:06.005712 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:13:06.005717 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:13:06.005995 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:13:06.006813 (+ 818us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:13:06.006858 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:13:06.008620 (+ 1762us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:13:06.010978 (+ 2358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:13:06.010986 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:13:06.011003 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:13:06.011318 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:13:06.011645 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:13:06.011649 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:13:06.011651 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:13:06.011803 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:13:06.012233 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:13:06.012241 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:13:06.012511 (+ 270us) client_negotiation.cc:769] Sending connection context
0805 23:13:06.012723 (+ 212us) client_negotiation.cc:240] Negotiation successful
0805 23:13:06.013253 (+ 530us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":167,"threads_started":1}
I20260805 23:13:06.015617 8608 heartbeater.cc:347] Connected to a master server at 127.25.92.254:45961
I20260805 23:13:06.015894 8608 heartbeater.cc:464] Registering TS with master...
I20260805 23:13:06.016526 8608 heartbeater.cc:511] Master 127.25.92.254:45961 requested a full tablet report, sending...
I20260805 23:13:06.018684 8270 ts_manager.cc:194] Registered new tserver with Master: 3f3cc8ff1e664a859a891b80666307f1 (127.25.92.194:37313)
I20260805 23:13:06.019373 8496 tablet_bootstrap.cc:492] T 010a96d4aeb243df9570dd9adf632526 P 3f3cc8ff1e664a859a891b80666307f1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260805 23:13:06.020000 8496 tablet_bootstrap.cc:492] T 010a96d4aeb243df9570dd9adf632526 P 3f3cc8ff1e664a859a891b80666307f1: Bootstrap complete.
I20260805 23:13:06.020020 8270 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.92.194@KRBTEST.COM'} at 127.25.92.194:48271
I20260805 23:13:06.021700 8496 ts_tablet_manager.cc:1397] T 010a96d4aeb243df9570dd9adf632526 P 3f3cc8ff1e664a859a891b80666307f1: Time spent bootstrapping tablet: real 0.038s user 0.022s sys 0.014s
I20260805 23:13:06.025378 8472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:06.019665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.194:37313 (local address 127.0.0.1:54834)
0805 23:13:06.019891 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0805 23:13:06.019915 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0805 23:13:06.020036 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:13:06.021905 (+ 1869us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:13:06.021908 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:13:06.021933 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:13:06.022037 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:13:06.022062 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:13:06.023257 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:13:06.023261 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:13:06.024005 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:13:06.024015 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:13:06.024858 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:13:06.024863 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:13:06.024957 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:13:06.024981 (+ 24us) client_negotiation.cc:769] Sending connection context
0805 23:13:06.025072 (+ 91us) client_negotiation.cc:240] Negotiation successful
0805 23:13:06.025183 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88}
I20260805 23:13:06.025399 8614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:06.019977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54834 (local address 127.25.92.194:37313)
0805 23:13:06.020957 (+ 980us) server_negotiation.cc:206] Beginning negotiation
0805 23:13:06.020968 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0805 23:13:06.020993 (+ 25us) server_negotiation.cc:407] Connection header received
0805 23:13:06.021070 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:13:06.021076 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:13:06.021666 (+ 590us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:13:06.022232 (+ 566us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:13:06.022342 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:13:06.023124 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:13:06.024159 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:13:06.024714 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:13:06.024845 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:13:06.025001 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0805 23:13:06.025104 (+ 103us) server_negotiation.cc:299] Negotiation successful
0805 23:13:06.025203 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":876,"thread_start_us":123,"threads_started":1}
I20260805 23:13:06.025218 8496 raft_consensus.cc:348] T 010a96d4aeb243df9570dd9adf632526 P 3f3cc8ff1e664a859a891b80666307f1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f3cc8ff1e664a859a891b80666307f1" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 37313 } } peers { permanent_uuid: "57ca592f72ed4d6f8fd9a07643f02fd5" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38153 } } peers { permanent_uuid: "9d7053e2a49447dc8f347c9188078e94" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 43069 } }
I20260805 23:13:06.026212 8496 raft_consensus.cc:729] T 010a96d4aeb243df9570dd9adf632526 P 3f3cc8ff1e664a859a891b80666307f1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f3cc8ff1e664a859a891b80666307f1, State: Initialized, Role: FOLLOWER
I20260805 23:13:06.026795 8496 consensus_queue.cc:260] T 010a96d4aeb243df9570dd9adf632526 P 3f3cc8ff1e664a859a891b80666307f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f3cc8ff1e664a859a891b80666307f1" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 37313 } } peers { permanent_uuid: "57ca592f72ed4d6f8fd9a07643f02fd5" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38153 } } peers { permanent_uuid: "9d7053e2a49447dc8f347c9188078e94" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 43069 } }
I20260805 23:13:06.026643 8540 tablet_service.cc:1514] Processing DeleteTablet for tablet 010a96d4aeb243df9570dd9adf632526 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:13:05 UTC) from {username='kudu', principal='kudu/127.25.92.254@KRBTEST.COM'} at 127.0.0.1:54834
I20260805 23:13:06.027573 8608 heartbeater.cc:503] Master 127.25.92.254:45961 was elected leader, sending a full tablet report...
I20260805 23:13:06.028101 8496 ts_tablet_manager.cc:1428] T 010a96d4aeb243df9570dd9adf632526 P 3f3cc8ff1e664a859a891b80666307f1: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
W20260805 23:13:06.029387 8255 catalog_manager.cc:4821] TS 3f3cc8ff1e664a859a891b80666307f1 (127.25.92.194:37313): delete failed for tablet 010a96d4aeb243df9570dd9adf632526 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 010a96d4aeb243df9570dd9adf632526 already in progress: opening tablet
I20260805 23:13:06.031302 8540 tablet_service.cc:1514] Processing DeleteTablet for tablet 010a96d4aeb243df9570dd9adf632526 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:13:05 UTC) from {username='kudu', principal='kudu/127.25.92.254@KRBTEST.COM'} at 127.0.0.1:54834
I20260805 23:13:06.031515 8616 tablet_replica.cc:330] stopping tablet replica
I20260805 23:13:06.031875 8616 raft_consensus.cc:2229] T 010a96d4aeb243df9570dd9adf632526 P 3f3cc8ff1e664a859a891b80666307f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 23:13:06.032137 8616 raft_consensus.cc:2258] T 010a96d4aeb243df9570dd9adf632526 P 3f3cc8ff1e664a859a891b80666307f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:13:06.033320 8616 ts_tablet_manager.cc:1905] T 010a96d4aeb243df9570dd9adf632526 P 3f3cc8ff1e664a859a891b80666307f1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:13:06.037261 8616 ts_tablet_manager.cc:1918] T 010a96d4aeb243df9570dd9adf632526 P 3f3cc8ff1e664a859a891b80666307f1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 23:13:06.037388 8616 log.cc:1196] T 010a96d4aeb243df9570dd9adf632526 P 3f3cc8ff1e664a859a891b80666307f1: Deleting WAL directory at /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-1/wal/wals/010a96d4aeb243df9570dd9adf632526
I20260805 23:13:06.037777 8616 ts_tablet_manager.cc:1939] T 010a96d4aeb243df9570dd9adf632526 P 3f3cc8ff1e664a859a891b80666307f1: Deleting consensus metadata
I20260805 23:13:06.039249 8255 catalog_manager.cc:4841] TS 3f3cc8ff1e664a859a891b80666307f1 (127.25.92.194:37313): tablet 010a96d4aeb243df9570dd9adf632526 (table test-table [id=a4d0cd17e8314dcc866a577685d67a66]) successfully deleted
W20260805 23:13:06.134552 8612 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:13:06.134886 8612 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:13:06.134963 8612 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:13:06.139228 8612 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:13:06.139328 8612 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:13:06.139427 8612 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.92.195
I20260805 23:13:06.145176 8612 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:41983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.195.keytab
--principal=kudu/127.25.92.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.92.195:38153
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.92.195
--webserver_port=44611
--tserver_master_addrs=127.25.92.254:45961
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8612
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.92.195
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:13:06.146426 8612 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:13:06.147503 8612 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:13:06.157856 8621 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:13:06.157843 8622 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:13:06.158042 8624 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:13:06.159646 8612 server_base.cc:1034] running on GCE node
I20260805 23:13:06.160142 8612 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:13:06.160835 8612 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:13:06.162010 8612 hybrid_clock.cc:648] HybridClock initialized: now 1785971586161977 us; error 48 us; skew 500 ppm
Aug 05 23:13:06 dist-test-slave-ss4n krb5kdc[7668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971586, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:13:06.165164 8612 init.cc:377] Logged in from keytab as kudu/127.25.92.195@KRBTEST.COM (short username kudu)
I20260805 23:13:06.166181 8612 webserver.cc:460] Webserver started at http://127.25.92.195:44611/ using document root <none> and password file <none>
I20260805 23:13:06.166718 8612 fs_manager.cc:359] Metadata directory not provided
I20260805 23:13:06.166787 8612 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:13:06.170847 8612 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260805 23:13:06.173260 8631 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:13:06.174763 8612 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:13:06.174872 8612 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-2/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-2/wal
uuid: "57ca592f72ed4d6f8fd9a07643f02fd5"
format_stamp: "Formatted at 2026-08-05 23:13:04 on dist-test-slave-ss4n"
server_key: "6148b95a70b48af2769e9dff78e44ce4"
server_key_iv: "27d0d2bb33804600190f62945429256a"
server_key_version: "encryptionkey@0"
I20260805 23:13:06.175446 8612 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:13:06.193432 8612 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:13:06.197126 8612 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:13:06.197326 8612 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:13:06.198117 8612 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:13:06.199004 8612 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:13:06.199056 8612 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:13:06.199107 8612 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:13:06.199123 8612 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:13:06.209159 8612 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.195:38153
I20260805 23:13:06.209190 8747 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.195:38153 every 8 connection(s)
I20260805 23:13:06.210319 8612 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-2/data/info.pb
I20260805 23:13:06.212733 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 8612
Aug 05 23:13:06 dist-test-slave-ss4n krb5kdc[7668](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971586, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.195@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM
I20260805 23:13:06.224095 8752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:06.213640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48554 (local address 127.25.92.254:45961)
0805 23:13:06.213926 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0805 23:13:06.213930 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:13:06.214203 (+ 273us) server_negotiation.cc:407] Connection header received
0805 23:13:06.214330 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:13:06.214335 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:13:06.214401 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:13:06.214525 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:13:06.215417 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:13:06.216236 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:13:06.217446 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:13:06.217780 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:13:06.217977 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:13:06.219638 (+ 1661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:13:06.219642 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:13:06.219646 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:13:06.219675 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:13:06.221552 (+ 1877us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:13:06.222158 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:13:06.222163 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:13:06.222165 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:13:06.222283 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:13:06.222613 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:13:06.222620 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:13:06.222623 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:13:06.222960 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:13:06.223180 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0805 23:13:06.223639 (+ 459us) server_negotiation.cc:299] Negotiation successful
0805 23:13:06.223823 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":26,"server-negotiator.queue_time_us":213,"thread_start_us":84,"threads_started":1}
I20260805 23:13:06.226869 8467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:06.212496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.195:53695 (local address 127.25.92.254:45961)
0805 23:13:06.212701 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0805 23:13:06.212709 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0805 23:13:06.213809 (+ 1100us) server_negotiation.cc:407] Connection header received
0805 23:13:06.215650 (+ 1841us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:13:06.215654 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:13:06.215701 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:13:06.215776 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:13:06.217246 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:13:06.217952 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:13:06.219159 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:13:06.219407 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:13:06.219533 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:13:06.222497 (+ 2964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:13:06.222500 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:13:06.222502 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:13:06.222524 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:13:06.224482 (+ 1958us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:13:06.225063 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:13:06.225068 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:13:06.225069 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:13:06.225168 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:13:06.225598 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:13:06.225601 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:13:06.225603 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:13:06.225812 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:13:06.225924 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0805 23:13:06.226554 (+ 630us) server_negotiation.cc:299] Negotiation successful
0805 23:13:06.226696 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":83}
I20260805 23:13:06.227758 8750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:06.212956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:45961 (local address 127.25.92.195:53695)
0805 23:13:06.213473 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0805 23:13:06.213695 (+ 222us) client_negotiation.cc:174] Beginning negotiation
0805 23:13:06.215359 (+ 1664us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:13:06.216011 (+ 652us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:13:06.216020 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:13:06.216503 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:13:06.217098 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:13:06.217114 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:13:06.218092 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:13:06.218095 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:13:06.219027 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:13:06.219037 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:13:06.219584 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:13:06.219588 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:13:06.219778 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:13:06.220557 (+ 779us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:13:06.220575 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:13:06.222275 (+ 1700us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:13:06.224618 (+ 2343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:13:06.224626 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:13:06.224651 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:13:06.224948 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:13:06.225267 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:13:06.225270 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:13:06.225272 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:13:06.225445 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:13:06.225923 (+ 478us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:13:06.225932 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:13:06.226279 (+ 347us) client_negotiation.cc:769] Sending connection context
0805 23:13:06.226497 (+ 218us) client_negotiation.cc:240] Negotiation successful
0805 23:13:06.226866 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":151,"threads_started":1}
I20260805 23:13:06.229000 8748 heartbeater.cc:347] Connected to a master server at 127.25.92.254:45961
I20260805 23:13:06.229307 8748 heartbeater.cc:464] Registering TS with master...
I20260805 23:13:06.229928 8748 heartbeater.cc:511] Master 127.25.92.254:45961 requested a full tablet report, sending...
I20260805 23:13:06.231220 8270 ts_manager.cc:194] Registered new tserver with Master: 57ca592f72ed4d6f8fd9a07643f02fd5 (127.25.92.195:38153)
I20260805 23:13:06.231895 8270 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.92.195@KRBTEST.COM'} at 127.25.92.195:53695
I20260805 23:13:06.238658 25971 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260805 23:13:06.243839 8467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:06.239349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48556 (local address 127.25.92.254:45961)
0805 23:13:06.239541 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0805 23:13:06.239545 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:13:06.239717 (+ 172us) server_negotiation.cc:407] Connection header received
0805 23:13:06.239872 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:13:06.239874 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:13:06.239985 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:13:06.240086 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 23:13:06.240366 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:13:06.240922 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:13:06.242080 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:13:06.242343 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:13:06.242448 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:13:06.242844 (+ 396us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 23:13:06.243377 (+ 533us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 23:13:06.243536 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0805 23:13:06.243605 (+ 69us) server_negotiation.cc:299] Negotiation successful
0805 23:13:06.243678 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260805 23:13:06.244746 8270 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:48556:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260805 23:13:06.248311 8270 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260805 23:13:06.257229 8540 tablet_service.cc:1467] Processing CreateTablet for tablet 33897e3f589e4555a289d1715f962f90 (DEFAULT_TABLE table=test-table [id=4b1af8c0c4ef46e4b2da462dc9c644a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:13:06.257448 8395 tablet_service.cc:1467] Processing CreateTablet for tablet 33897e3f589e4555a289d1715f962f90 (DEFAULT_TABLE table=test-table [id=4b1af8c0c4ef46e4b2da462dc9c644a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:13:06.257987 8540 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33897e3f589e4555a289d1715f962f90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:13:06.258025 8395 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33897e3f589e4555a289d1715f962f90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:13:06.261307 8496 tablet_bootstrap.cc:492] T 33897e3f589e4555a289d1715f962f90 P 3f3cc8ff1e664a859a891b80666307f1: Bootstrap starting.
I20260805 23:13:06.261891 8472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:06.256354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.195:38153 (local address 127.0.0.1:59340)
0805 23:13:06.256521 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0805 23:13:06.256537 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0805 23:13:06.256620 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:13:06.257827 (+ 1207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:13:06.257830 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:13:06.257849 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:13:06.257963 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:13:06.257972 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:13:06.259544 (+ 1572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:13:06.259551 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:13:06.260304 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:13:06.260313 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:13:06.261433 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:13:06.261437 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:13:06.261556 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:13:06.261601 (+ 45us) client_negotiation.cc:769] Sending connection context
0805 23:13:06.261654 (+ 53us) client_negotiation.cc:240] Negotiation successful
0805 23:13:06.261766 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87}
I20260805 23:13:06.262197 8496 tablet_bootstrap.cc:654] T 33897e3f589e4555a289d1715f962f90 P 3f3cc8ff1e664a859a891b80666307f1: Neither blocks nor log segments found. Creating new log.
I20260805 23:13:06.262537 8757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:06.256562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59340 (local address 127.25.92.195:38153)
0805 23:13:06.257082 (+ 520us) server_negotiation.cc:206] Beginning negotiation
0805 23:13:06.257098 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0805 23:13:06.257122 (+ 24us) server_negotiation.cc:407] Connection header received
0805 23:13:06.257247 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:13:06.257253 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:13:06.257552 (+ 299us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:13:06.258477 (+ 925us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:13:06.258661 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:13:06.259407 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:13:06.260442 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:13:06.261298 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:13:06.262113 (+ 815us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:13:06.262277 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0805 23:13:06.262332 (+ 55us) server_negotiation.cc:299] Negotiation successful
0805 23:13:06.262423 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":155,"threads_started":1}
I20260805 23:13:06.263337 8352 tablet_bootstrap.cc:492] T 33897e3f589e4555a289d1715f962f90 P 9d7053e2a49447dc8f347c9188078e94: Bootstrap starting.
I20260805 23:13:06.263983 8496 tablet_bootstrap.cc:492] T 33897e3f589e4555a289d1715f962f90 P 3f3cc8ff1e664a859a891b80666307f1: No bootstrap required, opened a new log
I20260805 23:13:06.264084 8496 ts_tablet_manager.cc:1397] T 33897e3f589e4555a289d1715f962f90 P 3f3cc8ff1e664a859a891b80666307f1: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260805 23:13:06.264201 8352 tablet_bootstrap.cc:654] T 33897e3f589e4555a289d1715f962f90 P 9d7053e2a49447dc8f347c9188078e94: Neither blocks nor log segments found. Creating new log.
I20260805 23:13:06.264530 8496 raft_consensus.cc:348] T 33897e3f589e4555a289d1715f962f90 P 3f3cc8ff1e664a859a891b80666307f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57ca592f72ed4d6f8fd9a07643f02fd5" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38153 } } peers { permanent_uuid: "3f3cc8ff1e664a859a891b80666307f1" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 37313 } } peers { permanent_uuid: "9d7053e2a49447dc8f347c9188078e94" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 43069 } }
I20260805 23:13:06.264671 8496 raft_consensus.cc:374] T 33897e3f589e4555a289d1715f962f90 P 3f3cc8ff1e664a859a891b80666307f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:13:06.264710 8496 raft_consensus.cc:729] T 33897e3f589e4555a289d1715f962f90 P 3f3cc8ff1e664a859a891b80666307f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f3cc8ff1e664a859a891b80666307f1, State: Initialized, Role: FOLLOWER
I20260805 23:13:06.264842 8496 consensus_queue.cc:260] T 33897e3f589e4555a289d1715f962f90 P 3f3cc8ff1e664a859a891b80666307f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57ca592f72ed4d6f8fd9a07643f02fd5" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38153 } } peers { permanent_uuid: "3f3cc8ff1e664a859a891b80666307f1" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 37313 } } peers { permanent_uuid: "9d7053e2a49447dc8f347c9188078e94" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 43069 } }
I20260805 23:13:06.265113 8496 ts_tablet_manager.cc:1428] T 33897e3f589e4555a289d1715f962f90 P 3f3cc8ff1e664a859a891b80666307f1: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260805 23:13:06.265400 8352 tablet_bootstrap.cc:492] T 33897e3f589e4555a289d1715f962f90 P 9d7053e2a49447dc8f347c9188078e94: No bootstrap required, opened a new log
I20260805 23:13:06.265592 8352 ts_tablet_manager.cc:1397] T 33897e3f589e4555a289d1715f962f90 P 9d7053e2a49447dc8f347c9188078e94: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260805 23:13:06.266078 8352 raft_consensus.cc:348] T 33897e3f589e4555a289d1715f962f90 P 9d7053e2a49447dc8f347c9188078e94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57ca592f72ed4d6f8fd9a07643f02fd5" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38153 } } peers { permanent_uuid: "3f3cc8ff1e664a859a891b80666307f1" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 37313 } } peers { permanent_uuid: "9d7053e2a49447dc8f347c9188078e94" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 43069 } }
I20260805 23:13:06.266186 8352 raft_consensus.cc:374] T 33897e3f589e4555a289d1715f962f90 P 9d7053e2a49447dc8f347c9188078e94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:13:06.266214 8352 raft_consensus.cc:729] T 33897e3f589e4555a289d1715f962f90 P 9d7053e2a49447dc8f347c9188078e94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d7053e2a49447dc8f347c9188078e94, State: Initialized, Role: FOLLOWER
I20260805 23:13:06.266350 8352 consensus_queue.cc:260] T 33897e3f589e4555a289d1715f962f90 P 9d7053e2a49447dc8f347c9188078e94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57ca592f72ed4d6f8fd9a07643f02fd5" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38153 } } peers { permanent_uuid: "3f3cc8ff1e664a859a891b80666307f1" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 37313 } } peers { permanent_uuid: "9d7053e2a49447dc8f347c9188078e94" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 43069 } }
I20260805 23:13:06.265897 8679 tablet_service.cc:1467] Processing CreateTablet for tablet 33897e3f589e4555a289d1715f962f90 (DEFAULT_TABLE table=test-table [id=4b1af8c0c4ef46e4b2da462dc9c644a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:13:06.266618 8352 ts_tablet_manager.cc:1428] T 33897e3f589e4555a289d1715f962f90 P 9d7053e2a49447dc8f347c9188078e94: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260805 23:13:06.267138 8679 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33897e3f589e4555a289d1715f962f90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:13:06.274667 8760 tablet_bootstrap.cc:492] T 33897e3f589e4555a289d1715f962f90 P 57ca592f72ed4d6f8fd9a07643f02fd5: Bootstrap starting.
I20260805 23:13:06.276891 8760 tablet_bootstrap.cc:654] T 33897e3f589e4555a289d1715f962f90 P 57ca592f72ed4d6f8fd9a07643f02fd5: Neither blocks nor log segments found. Creating new log.
I20260805 23:13:06.278159 8760 log.cc:824] T 33897e3f589e4555a289d1715f962f90 P 57ca592f72ed4d6f8fd9a07643f02fd5: Log is configured to *not* fsync() on all Append() calls
I20260805 23:13:06.280328 8760 tablet_bootstrap.cc:492] T 33897e3f589e4555a289d1715f962f90 P 57ca592f72ed4d6f8fd9a07643f02fd5: No bootstrap required, opened a new log
I20260805 23:13:06.280507 8760 ts_tablet_manager.cc:1397] T 33897e3f589e4555a289d1715f962f90 P 57ca592f72ed4d6f8fd9a07643f02fd5: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
W20260805 23:13:06.282390 8466 tablet.cc:2369] T 33897e3f589e4555a289d1715f962f90 P 9d7053e2a49447dc8f347c9188078e94: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260805 23:13:06.284340 8760 raft_consensus.cc:348] T 33897e3f589e4555a289d1715f962f90 P 57ca592f72ed4d6f8fd9a07643f02fd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57ca592f72ed4d6f8fd9a07643f02fd5" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38153 } } peers { permanent_uuid: "3f3cc8ff1e664a859a891b80666307f1" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 37313 } } peers { permanent_uuid: "9d7053e2a49447dc8f347c9188078e94" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 43069 } }
I20260805 23:13:06.284591 8760 raft_consensus.cc:374] T 33897e3f589e4555a289d1715f962f90 P 57ca592f72ed4d6f8fd9a07643f02fd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:13:06.284636 8760 raft_consensus.cc:729] T 33897e3f589e4555a289d1715f962f90 P 57ca592f72ed4d6f8fd9a07643f02fd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57ca592f72ed4d6f8fd9a07643f02fd5, State: Initialized, Role: FOLLOWER
I20260805 23:13:06.285063 8760 consensus_queue.cc:260] T 33897e3f589e4555a289d1715f962f90 P 57ca592f72ed4d6f8fd9a07643f02fd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57ca592f72ed4d6f8fd9a07643f02fd5" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38153 } } peers { permanent_uuid: "3f3cc8ff1e664a859a891b80666307f1" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 37313 } } peers { permanent_uuid: "9d7053e2a49447dc8f347c9188078e94" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 43069 } }
I20260805 23:13:06.285879 8748 heartbeater.cc:503] Master 127.25.92.254:45961 was elected leader, sending a full tablet report...
I20260805 23:13:06.286185 8760 ts_tablet_manager.cc:1428] T 33897e3f589e4555a289d1715f962f90 P 57ca592f72ed4d6f8fd9a07643f02fd5: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
I20260805 23:13:06.387331 8615 raft_consensus.cc:482] T 33897e3f589e4555a289d1715f962f90 P 3f3cc8ff1e664a859a891b80666307f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 23:13:06.387614 8615 raft_consensus.cc:504] T 33897e3f589e4555a289d1715f962f90 P 3f3cc8ff1e664a859a891b80666307f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57ca592f72ed4d6f8fd9a07643f02fd5" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38153 } } peers { permanent_uuid: "3f3cc8ff1e664a859a891b80666307f1" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 37313 } } peers { permanent_uuid: "9d7053e2a49447dc8f347c9188078e94" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 43069 } }
I20260805 23:13:06.388849 8615 leader_election.cc:288] T 33897e3f589e4555a289d1715f962f90 P 3f3cc8ff1e664a859a891b80666307f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57ca592f72ed4d6f8fd9a07643f02fd5 (127.25.92.195:38153), 9d7053e2a49447dc8f347c9188078e94 (127.25.92.193:43069)
I20260805 23:13:06.393148 8610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:06.389216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.193:43069 (local address 127.25.92.194:33847)
0805 23:13:06.389371 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0805 23:13:06.389389 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0805 23:13:06.389538 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:13:06.389991 (+ 453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:13:06.389994 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:13:06.390012 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:13:06.390167 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:13:06.390175 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:13:06.391095 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:13:06.391100 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:13:06.391908 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:13:06.391917 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:13:06.392668 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:13:06.392672 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:13:06.392800 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:13:06.392830 (+ 30us) client_negotiation.cc:769] Sending connection context
0805 23:13:06.392918 (+ 88us) client_negotiation.cc:240] Negotiation successful
0805 23:13:06.393000 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73}
I20260805 23:13:06.393153 8763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:06.389384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.194:33847 (local address 127.25.92.193:43069)
0805 23:13:06.389755 (+ 371us) server_negotiation.cc:206] Beginning negotiation
0805 23:13:06.389758 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 23:13:06.389772 (+ 14us) server_negotiation.cc:407] Connection header received
0805 23:13:06.389822 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:13:06.389824 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:13:06.389863 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:13:06.389976 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:13:06.390305 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:13:06.390909 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:13:06.392109 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:13:06.392538 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:13:06.392675 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:13:06.392752 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0805 23:13:06.392951 (+ 199us) server_negotiation.cc:299] Negotiation successful
0805 23:13:06.393038 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":218,"threads_started":1}
I20260805 23:13:06.393420 8757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:06.389398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.194:60771 (local address 127.25.92.195:38153)
0805 23:13:06.389589 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0805 23:13:06.389597 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0805 23:13:06.390078 (+ 481us) server_negotiation.cc:407] Connection header received
0805 23:13:06.390214 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:13:06.390218 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:13:06.390272 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:13:06.390367 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:13:06.390688 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:13:06.391189 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:13:06.392325 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:13:06.392787 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:13:06.392941 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:13:06.392998 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0805 23:13:06.393209 (+ 211us) server_negotiation.cc:299] Negotiation successful
0805 23:13:06.393288 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260805 23:13:06.393453 8764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:06.389216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.195:38153 (local address 127.25.92.194:60771)
0805 23:13:06.389968 (+ 752us) negotiation.cc:107] Waiting for socket to connect
0805 23:13:06.389981 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0805 23:13:06.390113 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:13:06.390380 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:13:06.390383 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:13:06.390399 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:13:06.390547 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:13:06.390557 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:13:06.391316 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:13:06.391321 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:13:06.392179 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:13:06.392189 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:13:06.392951 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:13:06.392955 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:13:06.393072 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:13:06.393091 (+ 19us) client_negotiation.cc:769] Sending connection context
0805 23:13:06.393192 (+ 101us) client_negotiation.cc:240] Negotiation successful
0805 23:13:06.393282 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":670,"mutex_wait_us":405,"thread_start_us":138,"threads_started":1}
I20260805 23:13:06.394258 8700 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33897e3f589e4555a289d1715f962f90" candidate_uuid: "3f3cc8ff1e664a859a891b80666307f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57ca592f72ed4d6f8fd9a07643f02fd5" is_pre_election: true
I20260805 23:13:06.394610 8700 raft_consensus.cc:2454] T 33897e3f589e4555a289d1715f962f90 P 57ca592f72ed4d6f8fd9a07643f02fd5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f3cc8ff1e664a859a891b80666307f1 in term 0.
I20260805 23:13:06.395058 8417 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33897e3f589e4555a289d1715f962f90" candidate_uuid: "3f3cc8ff1e664a859a891b80666307f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d7053e2a49447dc8f347c9188078e94" is_pre_election: true
I20260805 23:13:06.395186 8491 leader_election.cc:302] T 33897e3f589e4555a289d1715f962f90 P 3f3cc8ff1e664a859a891b80666307f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f3cc8ff1e664a859a891b80666307f1, 57ca592f72ed4d6f8fd9a07643f02fd5; no voters:
I20260805 23:13:06.395354 8417 raft_consensus.cc:2454] T 33897e3f589e4555a289d1715f962f90 P 9d7053e2a49447dc8f347c9188078e94 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f3cc8ff1e664a859a891b80666307f1 in term 0.
I20260805 23:13:06.395563 8615 raft_consensus.cc:2781] T 33897e3f589e4555a289d1715f962f90 P 3f3cc8ff1e664a859a891b80666307f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260805 23:13:06.395673 8615 raft_consensus.cc:482] T 33897e3f589e4555a289d1715f962f90 P 3f3cc8ff1e664a859a891b80666307f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260805 23:13:06.395730 8615 raft_consensus.cc:3037] T 33897e3f589e4555a289d1715f962f90 P 3f3cc8ff1e664a859a891b80666307f1 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:13:06.396826 8615 raft_consensus.cc:504] T 33897e3f589e4555a289d1715f962f90 P 3f3cc8ff1e664a859a891b80666307f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57ca592f72ed4d6f8fd9a07643f02fd5" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38153 } } peers { permanent_uuid: "3f3cc8ff1e664a859a891b80666307f1" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 37313 } } peers { permanent_uuid: "9d7053e2a49447dc8f347c9188078e94" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 43069 } }
I20260805 23:13:06.397220 8615 leader_election.cc:288] T 33897e3f589e4555a289d1715f962f90 P 3f3cc8ff1e664a859a891b80666307f1 [CANDIDATE]: Term 1 election: Requested vote from peers 57ca592f72ed4d6f8fd9a07643f02fd5 (127.25.92.195:38153), 9d7053e2a49447dc8f347c9188078e94 (127.25.92.193:43069)
I20260805 23:13:06.397670 8700 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33897e3f589e4555a289d1715f962f90" candidate_uuid: "3f3cc8ff1e664a859a891b80666307f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57ca592f72ed4d6f8fd9a07643f02fd5"
I20260805 23:13:06.397675 8417 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33897e3f589e4555a289d1715f962f90" candidate_uuid: "3f3cc8ff1e664a859a891b80666307f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d7053e2a49447dc8f347c9188078e94"
I20260805 23:13:06.397811 8417 raft_consensus.cc:3037] T 33897e3f589e4555a289d1715f962f90 P 9d7053e2a49447dc8f347c9188078e94 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:13:06.397811 8700 raft_consensus.cc:3037] T 33897e3f589e4555a289d1715f962f90 P 57ca592f72ed4d6f8fd9a07643f02fd5 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:13:06.399185 8700 raft_consensus.cc:2454] T 33897e3f589e4555a289d1715f962f90 P 57ca592f72ed4d6f8fd9a07643f02fd5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f3cc8ff1e664a859a891b80666307f1 in term 1.
I20260805 23:13:06.399183 8417 raft_consensus.cc:2454] T 33897e3f589e4555a289d1715f962f90 P 9d7053e2a49447dc8f347c9188078e94 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f3cc8ff1e664a859a891b80666307f1 in term 1.
I20260805 23:13:06.399534 8493 leader_election.cc:302] T 33897e3f589e4555a289d1715f962f90 P 3f3cc8ff1e664a859a891b80666307f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f3cc8ff1e664a859a891b80666307f1, 9d7053e2a49447dc8f347c9188078e94; no voters:
I20260805 23:13:06.399755 8615 raft_consensus.cc:2781] T 33897e3f589e4555a289d1715f962f90 P 3f3cc8ff1e664a859a891b80666307f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:13:06.400025 8615 raft_consensus.cc:686] T 33897e3f589e4555a289d1715f962f90 P 3f3cc8ff1e664a859a891b80666307f1 [term 1 LEADER]: Becoming Leader. State: Replica: 3f3cc8ff1e664a859a891b80666307f1, State: Running, Role: LEADER
I20260805 23:13:06.400300 8615 consensus_queue.cc:237] T 33897e3f589e4555a289d1715f962f90 P 3f3cc8ff1e664a859a891b80666307f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57ca592f72ed4d6f8fd9a07643f02fd5" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38153 } } peers { permanent_uuid: "3f3cc8ff1e664a859a891b80666307f1" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 37313 } } peers { permanent_uuid: "9d7053e2a49447dc8f347c9188078e94" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 43069 } }
I20260805 23:13:06.403868 8268 catalog_manager.cc:5495] T 33897e3f589e4555a289d1715f962f90 P 3f3cc8ff1e664a859a891b80666307f1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3f3cc8ff1e664a859a891b80666307f1 (127.25.92.194). New cstate: current_term: 1 leader_uuid: "3f3cc8ff1e664a859a891b80666307f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57ca592f72ed4d6f8fd9a07643f02fd5" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f3cc8ff1e664a859a891b80666307f1" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 37313 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d7053e2a49447dc8f347c9188078e94" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 43069 } health_report { overall_health: UNKNOWN } } }
I20260805 23:13:06.423130 8614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:06.419157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54848 (local address 127.25.92.194:37313)
0805 23:13:06.419331 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0805 23:13:06.419336 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:13:06.419360 (+ 24us) server_negotiation.cc:407] Connection header received
0805 23:13:06.419444 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:13:06.419447 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:13:06.419524 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:13:06.419658 (+ 134us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 23:13:06.419922 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:13:06.420482 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:13:06.421424 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:13:06.421729 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:13:06.421851 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:13:06.422162 (+ 311us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 23:13:06.422662 (+ 500us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 23:13:06.422797 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0805 23:13:06.422882 (+ 85us) server_negotiation.cc:299] Negotiation successful
0805 23:13:06.422965 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260805 23:13:06.427985 8700 raft_consensus.cc:1261] T 33897e3f589e4555a289d1715f962f90 P 57ca592f72ed4d6f8fd9a07643f02fd5 [term 1 FOLLOWER]: Refusing update from remote peer 3f3cc8ff1e664a859a891b80666307f1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:13:06.428194 8417 raft_consensus.cc:1261] T 33897e3f589e4555a289d1715f962f90 P 9d7053e2a49447dc8f347c9188078e94 [term 1 FOLLOWER]: Refusing update from remote peer 3f3cc8ff1e664a859a891b80666307f1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:13:06.428994 8615 consensus_queue.cc:1035] T 33897e3f589e4555a289d1715f962f90 P 3f3cc8ff1e664a859a891b80666307f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57ca592f72ed4d6f8fd9a07643f02fd5" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:13:06.429347 8765 consensus_queue.cc:1035] T 33897e3f589e4555a289d1715f962f90 P 3f3cc8ff1e664a859a891b80666307f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d7053e2a49447dc8f347c9188078e94" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 43069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:13:06.435956 8768 mvcc.cc:205] Tried to move back new op lower bound from 7315339618000609280 to 7315339617898119168. Current Snapshot: MvccSnapshot[applied={T|T < 7315339618000609280}]
I20260805 23:13:06.438176 8769 mvcc.cc:205] Tried to move back new op lower bound from 7315339618000609280 to 7315339617898119168. Current Snapshot: MvccSnapshot[applied={T|T < 7315339618000609280}]
I20260805 23:13:06.444866 8270 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:48556:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:13:06.445088 8270 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:48556:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:13:06.447839 8270 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cea136ae38a74d03b8dfdfc98787079d: Sending DeleteTablet for 3 replicas of tablet 33897e3f589e4555a289d1715f962f90
I20260805 23:13:06.448635 8540 tablet_service.cc:1514] Processing DeleteTablet for tablet 33897e3f589e4555a289d1715f962f90 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:13:06 UTC) from {username='kudu', principal='kudu/127.25.92.254@KRBTEST.COM'} at 127.0.0.1:54834
I20260805 23:13:06.448673 8679 tablet_service.cc:1514] Processing DeleteTablet for tablet 33897e3f589e4555a289d1715f962f90 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:13:06 UTC) from {username='kudu', principal='kudu/127.25.92.254@KRBTEST.COM'} at 127.0.0.1:59340
I20260805 23:13:06.448778 8395 tablet_service.cc:1514] Processing DeleteTablet for tablet 33897e3f589e4555a289d1715f962f90 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:13:06 UTC) from {username='kudu', principal='kudu/127.25.92.254@KRBTEST.COM'} at 127.0.0.1:43076
I20260805 23:13:06.448837 8616 tablet_replica.cc:330] T 33897e3f589e4555a289d1715f962f90 P 3f3cc8ff1e664a859a891b80666307f1: stopping tablet replica
I20260805 23:13:06.448949 8616 raft_consensus.cc:2229] T 33897e3f589e4555a289d1715f962f90 P 3f3cc8ff1e664a859a891b80666307f1 [term 1 LEADER]: Raft consensus shutting down.
I20260805 23:13:06.449292 8616 raft_consensus.cc:2258] T 33897e3f589e4555a289d1715f962f90 P 3f3cc8ff1e664a859a891b80666307f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:13:06.449406 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 8310
I20260805 23:13:06.450440 8779 tablet_replica.cc:330] T 33897e3f589e4555a289d1715f962f90 P 57ca592f72ed4d6f8fd9a07643f02fd5: stopping tablet replica
I20260805 23:13:06.450760 8779 raft_consensus.cc:2229] T 33897e3f589e4555a289d1715f962f90 P 57ca592f72ed4d6f8fd9a07643f02fd5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 23:13:06.451157 8779 raft_consensus.cc:2258] T 33897e3f589e4555a289d1715f962f90 P 57ca592f72ed4d6f8fd9a07643f02fd5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:13:06.451244 8616 ts_tablet_manager.cc:1905] T 33897e3f589e4555a289d1715f962f90 P 3f3cc8ff1e664a859a891b80666307f1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:13:06.452826 8779 ts_tablet_manager.cc:1905] T 33897e3f589e4555a289d1715f962f90 P 57ca592f72ed4d6f8fd9a07643f02fd5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:13:06.453984 8616 ts_tablet_manager.cc:1918] T 33897e3f589e4555a289d1715f962f90 P 3f3cc8ff1e664a859a891b80666307f1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 23:13:06.454066 8616 log.cc:1196] T 33897e3f589e4555a289d1715f962f90 P 3f3cc8ff1e664a859a891b80666307f1: Deleting WAL directory at /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-1/wal/wals/33897e3f589e4555a289d1715f962f90
I20260805 23:13:06.454340 8616 ts_tablet_manager.cc:1939] T 33897e3f589e4555a289d1715f962f90 P 3f3cc8ff1e664a859a891b80666307f1: Deleting consensus metadata
I20260805 23:13:06.455245 8255 catalog_manager.cc:4841] TS 3f3cc8ff1e664a859a891b80666307f1 (127.25.92.194:37313): tablet 33897e3f589e4555a289d1715f962f90 (table test-table [id=4b1af8c0c4ef46e4b2da462dc9c644a2]) successfully deleted
I20260805 23:13:06.455665 8779 ts_tablet_manager.cc:1918] T 33897e3f589e4555a289d1715f962f90 P 57ca592f72ed4d6f8fd9a07643f02fd5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 23:13:06.455801 8779 log.cc:1196] T 33897e3f589e4555a289d1715f962f90 P 57ca592f72ed4d6f8fd9a07643f02fd5: Deleting WAL directory at /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-2/wal/wals/33897e3f589e4555a289d1715f962f90
I20260805 23:13:06.456106 8779 ts_tablet_manager.cc:1939] T 33897e3f589e4555a289d1715f962f90 P 57ca592f72ed4d6f8fd9a07643f02fd5: Deleting consensus metadata
I20260805 23:13:06.457592 8254 catalog_manager.cc:4841] TS 57ca592f72ed4d6f8fd9a07643f02fd5 (127.25.92.195:38153): tablet 33897e3f589e4555a289d1715f962f90 (table test-table [id=4b1af8c0c4ef46e4b2da462dc9c644a2]) successfully deleted
W20260805 23:13:06.460520 8256 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.25.92.193:43069 (error 108)
I20260805 23:13:06.461127 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 8469
I20260805 23:13:06.462733 8472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:06.462436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.193:43069 (local address 127.0.0.1:43082)
0805 23:13:06.462570 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0805 23:13:06.462656 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.193:43069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260805 23:13:06.462952 8256 catalog_manager.cc:4584] TS 9d7053e2a49447dc8f347c9188078e94 (127.25.92.193:43069): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 33897e3f589e4555a289d1715f962f90: Network error: Client connection negotiation failed: client connection to 127.25.92.193:43069: connect: Connection refused (error 111)
I20260805 23:13:06.468513 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 8612
I20260805 23:13:06.475767 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 8236
I20260805 23:13:06.483455 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.254:45961
--webserver_interface=127.25.92.254
--webserver_port=46373
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab
--principal=kudu/127.25.92.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:41983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.92.254:45961
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:13:06.605317 8781 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:13:06.605775 8781 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:13:06.605878 8781 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:13:06.610846 8781 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:13:06.610960 8781 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:13:06.611001 8781 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:13:06.611049 8781 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:13:06.611091 8781 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:13:06.617447 8781 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:41983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.92.254:45961
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab
--principal=kudu/127.25.92.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.254:45961
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.92.254
--webserver_port=46373
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8781
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:13:06.618901 8781 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:13:06.620186 8781 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:13:06.629844 8790 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:13:06.629835 8788 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:13:06.629926 8781 server_base.cc:1034] running on GCE node
W20260805 23:13:06.629848 8787 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:13:06.630934 8781 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:13:06.632167 8781 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:13:06.633366 8781 hybrid_clock.cc:648] HybridClock initialized: now 1785971586633325 us; error 55 us; skew 500 ppm
Aug 05 23:13:06 dist-test-slave-ss4n krb5kdc[7668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971586, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:13:06.636776 8781 init.cc:377] Logged in from keytab as kudu/127.25.92.254@KRBTEST.COM (short username kudu)
I20260805 23:13:06.637940 8781 webserver.cc:460] Webserver started at http://127.25.92.254:46373/ using document root <none> and password file <none>
I20260805 23:13:06.638577 8781 fs_manager.cc:359] Metadata directory not provided
I20260805 23:13:06.638650 8781 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:13:06.643177 8781 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260805 23:13:06.645988 8797 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:13:06.647835 8781 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:13:06.648016 8781 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/master-0/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/master-0/wal
uuid: "cea136ae38a74d03b8dfdfc98787079d"
format_stamp: "Formatted at 2026-08-05 23:13:03 on dist-test-slave-ss4n"
server_key: "f5eff3ff5f4038c6d9d0ec33d02b3020"
server_key_iv: "4787f4b860dd10e3df762e1ae1ff1bad"
server_key_version: "encryptionkey@0"
I20260805 23:13:06.648679 8781 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:13:06.663765 8781 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:13:06.667280 8781 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:13:06.667507 8781 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:13:06.675309 8781 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.254:45961
I20260805 23:13:06.675688 8849 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.254:45961 every 8 connection(s)
I20260805 23:13:06.676878 8781 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
I20260805 23:13:06.681703 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 8781
I20260805 23:13:06.682304 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.92.193:43069
--local_ip_for_outbound_sockets=127.25.92.193
--tserver_master_addrs=127.25.92.254:45961
--webserver_port=36179
--webserver_interface=127.25.92.193
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.193.keytab
--principal=kudu/127.25.92.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:41983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260805 23:13:06.683969 8850 sys_catalog.cc:263] Verifying existing consensus state
I20260805 23:13:06.687475 8850 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cea136ae38a74d03b8dfdfc98787079d: Bootstrap starting.
I20260805 23:13:06.713943 8850 log.cc:824] T 00000000000000000000000000000000 P cea136ae38a74d03b8dfdfc98787079d: Log is configured to *not* fsync() on all Append() calls
I20260805 23:13:06.731565 8850 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cea136ae38a74d03b8dfdfc98787079d: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260805 23:13:06.732023 8850 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cea136ae38a74d03b8dfdfc98787079d: Bootstrap complete.
I20260805 23:13:06.736423 8850 raft_consensus.cc:348] T 00000000000000000000000000000000 P cea136ae38a74d03b8dfdfc98787079d [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cea136ae38a74d03b8dfdfc98787079d" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 45961 } }
I20260805 23:13:06.736862 8850 raft_consensus.cc:729] T 00000000000000000000000000000000 P cea136ae38a74d03b8dfdfc98787079d [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: cea136ae38a74d03b8dfdfc98787079d, State: Initialized, Role: FOLLOWER
I20260805 23:13:06.737363 8850 consensus_queue.cc:260] T 00000000000000000000000000000000 P cea136ae38a74d03b8dfdfc98787079d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cea136ae38a74d03b8dfdfc98787079d" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 45961 } }
I20260805 23:13:06.737542 8850 raft_consensus.cc:388] T 00000000000000000000000000000000 P cea136ae38a74d03b8dfdfc98787079d [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:13:06.737634 8850 raft_consensus.cc:482] T 00000000000000000000000000000000 P cea136ae38a74d03b8dfdfc98787079d [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:13:06.737753 8850 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cea136ae38a74d03b8dfdfc98787079d [term 2 FOLLOWER]: Advancing to term 3
I20260805 23:13:06.739820 8850 raft_consensus.cc:504] T 00000000000000000000000000000000 P cea136ae38a74d03b8dfdfc98787079d [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cea136ae38a74d03b8dfdfc98787079d" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 45961 } }
I20260805 23:13:06.740170 8850 leader_election.cc:302] T 00000000000000000000000000000000 P cea136ae38a74d03b8dfdfc98787079d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cea136ae38a74d03b8dfdfc98787079d; no voters:
I20260805 23:13:06.740520 8850 leader_election.cc:288] T 00000000000000000000000000000000 P cea136ae38a74d03b8dfdfc98787079d [CANDIDATE]: Term 3 election: Requested vote from peers
I20260805 23:13:06.740608 8854 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cea136ae38a74d03b8dfdfc98787079d [term 3 FOLLOWER]: Leader election won for term 3
I20260805 23:13:06.740836 8854 raft_consensus.cc:686] T 00000000000000000000000000000000 P cea136ae38a74d03b8dfdfc98787079d [term 3 LEADER]: Becoming Leader. State: Replica: cea136ae38a74d03b8dfdfc98787079d, State: Running, Role: LEADER
I20260805 23:13:06.741140 8854 consensus_queue.cc:237] T 00000000000000000000000000000000 P cea136ae38a74d03b8dfdfc98787079d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cea136ae38a74d03b8dfdfc98787079d" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 45961 } }
I20260805 23:13:06.741812 8850 sys_catalog.cc:564] T 00000000000000000000000000000000 P cea136ae38a74d03b8dfdfc98787079d [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:13:06.742612 8855 sys_catalog.cc:455] T 00000000000000000000000000000000 P cea136ae38a74d03b8dfdfc98787079d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "cea136ae38a74d03b8dfdfc98787079d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cea136ae38a74d03b8dfdfc98787079d" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 45961 } } }
I20260805 23:13:06.742861 8855 sys_catalog.cc:458] T 00000000000000000000000000000000 P cea136ae38a74d03b8dfdfc98787079d [sys.catalog]: This master's current role is: LEADER
I20260805 23:13:06.743316 8856 sys_catalog.cc:455] T 00000000000000000000000000000000 P cea136ae38a74d03b8dfdfc98787079d [sys.catalog]: SysCatalogTable state changed. Reason: New leader cea136ae38a74d03b8dfdfc98787079d. Latest consensus state: current_term: 3 leader_uuid: "cea136ae38a74d03b8dfdfc98787079d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cea136ae38a74d03b8dfdfc98787079d" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 45961 } } }
I20260805 23:13:06.743484 8856 sys_catalog.cc:458] T 00000000000000000000000000000000 P cea136ae38a74d03b8dfdfc98787079d [sys.catalog]: This master's current role is: LEADER
I20260805 23:13:06.747033 8870 catalog_manager.cc:1259] Loaded cluster ID: 31a5fac73c5b4a22a381a0c99d3ea80a
I20260805 23:13:06.747099 8870 catalog_manager.cc:1552] T 00000000000000000000000000000000 P cea136ae38a74d03b8dfdfc98787079d: loading cluster ID for follower catalog manager: success
W20260805 23:13:06.748164 8870 catalog_manager.cc:1576] T 00000000000000000000000000000000 P cea136ae38a74d03b8dfdfc98787079d: acquiring CA information for follower catalog manager: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
W20260805 23:13:06.748240 8870 catalog_manager.cc:873] Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO: failed to prepare follower catalog manager, will retry
I20260805 23:13:06.748310 8863 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:13:06.751163 8863 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:13:06.751765 8863 catalog_manager.cc:1259] Loaded cluster ID: 31a5fac73c5b4a22a381a0c99d3ea80a
I20260805 23:13:06.751824 8863 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260805 23:13:06.752488 8863 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f932cfe19ad google::LogMessage::Fail() at ??:0
@ 0x7f932cfe488a google::LogMessage::SendToLog() at ??:0
@ 0x7f932cfe14de google::LogMessage::Flush() at ??:0
@ 0x7f932cfe2a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f9337920fdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f9337921d2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f933791e579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f9337962640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f932f2333d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f932f233c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f932f2353f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f932f224512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f93308686db start_thread at ??:0
@ 0x7f932badf61f clone at ??:0
W20260805 23:13:06.861007 8852 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:13:06.861424 8852 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:13:06.861586 8852 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:13:06.866439 8852 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:13:06.866515 8852 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:13:06.866622 8852 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.92.193
I20260805 23:13:06.872967 8852 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:41983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.193.keytab
--principal=kudu/127.25.92.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.92.193:43069
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.92.193
--webserver_port=36179
--tserver_master_addrs=127.25.92.254:45961
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8852
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.92.193
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:13:06.874253 8852 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:13:06.875381 8852 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:13:06.886252 8894 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:13:06.886266 8893 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:13:06.886304 8896 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:13:06.886808 8852 server_base.cc:1034] running on GCE node
I20260805 23:13:06.887308 8852 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:13:06.888115 8852 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:13:06.889324 8852 hybrid_clock.cc:648] HybridClock initialized: now 1785971586889283 us; error 51 us; skew 500 ppm
Aug 05 23:13:06 dist-test-slave-ss4n krb5kdc[7668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971586, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:13:06.892817 8852 init.cc:377] Logged in from keytab as kudu/127.25.92.193@KRBTEST.COM (short username kudu)
I20260805 23:13:06.893875 8852 webserver.cc:460] Webserver started at http://127.25.92.193:36179/ using document root <none> and password file <none>
I20260805 23:13:06.894486 8852 fs_manager.cc:359] Metadata directory not provided
I20260805 23:13:06.894552 8852 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:13:06.898785 8852 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.004s
I20260805 23:13:06.901221 8903 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:13:06.902964 8852 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260805 23:13:06.903168 8852 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-0/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-0/wal
uuid: "9d7053e2a49447dc8f347c9188078e94"
format_stamp: "Formatted at 2026-08-05 23:13:04 on dist-test-slave-ss4n"
server_key: "d5e2b7c396a15bd3a8b77add6d817f32"
server_key_iv: "f0c4bb51274444f28601105a60511d8d"
server_key_version: "encryptionkey@0"
I20260805 23:13:06.903859 8852 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:13:06.926612 8852 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:13:06.930855 8852 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:13:06.931135 8852 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:13:06.932014 8852 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:13:06.933441 8910 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260805 23:13:06.939210 8852 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260805 23:13:06.939332 8852 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260805 23:13:06.939450 8852 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260805 23:13:06.941587 8852 ts_tablet_manager.cc:610] Registered 1 tablets
I20260805 23:13:06.941681 8852 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.002s
I20260805 23:13:06.941949 8910 tablet_bootstrap.cc:492] T 33897e3f589e4555a289d1715f962f90 P 9d7053e2a49447dc8f347c9188078e94: Bootstrap starting.
I20260805 23:13:06.954334 8852 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.193:43069
I20260805 23:13:06.954655 9023 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.193:43069 every 8 connection(s)
I20260805 23:13:06.955564 8852 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-0/data/info.pb
I20260805 23:13:06.963140 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 8852
I20260805 23:13:06.963606 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.92.194:37313
--local_ip_for_outbound_sockets=127.25.92.194
--tserver_master_addrs=127.25.92.254:45961
--webserver_port=44697
--webserver_interface=127.25.92.194
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.194.keytab
--principal=kudu/127.25.92.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:41983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260805 23:13:06.964792 8910 log.cc:824] T 33897e3f589e4555a289d1715f962f90 P 9d7053e2a49447dc8f347c9188078e94: Log is configured to *not* fsync() on all Append() calls
I20260805 23:13:06.974674 8910 tablet_bootstrap.cc:492] T 33897e3f589e4555a289d1715f962f90 P 9d7053e2a49447dc8f347c9188078e94: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260805 23:13:06.975149 8910 tablet_bootstrap.cc:492] T 33897e3f589e4555a289d1715f962f90 P 9d7053e2a49447dc8f347c9188078e94: Bootstrap complete.
I20260805 23:13:06.976605 8910 ts_tablet_manager.cc:1397] T 33897e3f589e4555a289d1715f962f90 P 9d7053e2a49447dc8f347c9188078e94: Time spent bootstrapping tablet: real 0.035s user 0.027s sys 0.004s
I20260805 23:13:06.979961 8910 raft_consensus.cc:348] T 33897e3f589e4555a289d1715f962f90 P 9d7053e2a49447dc8f347c9188078e94 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57ca592f72ed4d6f8fd9a07643f02fd5" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38153 } } peers { permanent_uuid: "3f3cc8ff1e664a859a891b80666307f1" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 37313 } } peers { permanent_uuid: "9d7053e2a49447dc8f347c9188078e94" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 43069 } }
I20260805 23:13:06.980535 8910 raft_consensus.cc:729] T 33897e3f589e4555a289d1715f962f90 P 9d7053e2a49447dc8f347c9188078e94 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d7053e2a49447dc8f347c9188078e94, State: Initialized, Role: FOLLOWER
I20260805 23:13:06.981093 8910 consensus_queue.cc:260] T 33897e3f589e4555a289d1715f962f90 P 9d7053e2a49447dc8f347c9188078e94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57ca592f72ed4d6f8fd9a07643f02fd5" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38153 } } peers { permanent_uuid: "3f3cc8ff1e664a859a891b80666307f1" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 37313 } } peers { permanent_uuid: "9d7053e2a49447dc8f347c9188078e94" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 43069 } }
I20260805 23:13:06.982246 8910 ts_tablet_manager.cc:1428] T 33897e3f589e4555a289d1715f962f90 P 9d7053e2a49447dc8f347c9188078e94: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
W20260805 23:13:06.993743 9026 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:13:06.957987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:45961 (local address 127.25.92.193:33819)
0805 23:13:06.958544 (+ 557us) negotiation.cc:107] Waiting for socket to connect
0805 23:13:06.958608 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0805 23:13:06.959549 (+ 941us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:13:06.992734 (+ 33185us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:45961: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":162,"threads_started":1}
W20260805 23:13:06.994402 9024 heartbeater.cc:644] Failed to heartbeat to 127.25.92.254:45961 (0 consecutive failures): Network error: Failed to ping master at 127.25.92.254:45961: Client connection negotiation failed: client connection to 127.25.92.254:45961: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260805 23:13:06.995348 9026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:06.995062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:45961 (local address 127.25.92.193:55089)
0805 23:13:06.995211 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0805 23:13:06.995282 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:45961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260805 23:13:06.996299 9026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:06.996027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:45961 (local address 127.25.92.193:51055)
0805 23:13:06.996183 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0805 23:13:06.996240 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:45961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
W20260805 23:13:06.996639 9024 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260805 23:13:07.088399 9028 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:13:07.088716 9028 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:13:07.088802 9028 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:13:07.092891 9028 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:13:07.092996 9028 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:13:07.093091 9028 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.92.194
I20260805 23:13:07.098827 9028 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:41983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.194.keytab
--principal=kudu/127.25.92.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.92.194:37313
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.92.194
--webserver_port=44697
--tserver_master_addrs=127.25.92.254:45961
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9028
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.92.194
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:13:07.100059 9028 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:13:07.101145 9028 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:13:07.111361 9035 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:13:07.111371 9038 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:13:07.111588 9028 server_base.cc:1034] running on GCE node
W20260805 23:13:07.111361 9036 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:13:07.112195 9028 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:13:07.112905 9028 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:13:07.114099 9028 hybrid_clock.cc:648] HybridClock initialized: now 1785971587114064 us; error 49 us; skew 500 ppm
Aug 05 23:13:07 dist-test-slave-ss4n krb5kdc[7668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971587, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:13:07.117003 9028 init.cc:377] Logged in from keytab as kudu/127.25.92.194@KRBTEST.COM (short username kudu)
I20260805 23:13:07.118022 9028 webserver.cc:460] Webserver started at http://127.25.92.194:44697/ using document root <none> and password file <none>
I20260805 23:13:07.118662 9028 fs_manager.cc:359] Metadata directory not provided
I20260805 23:13:07.118768 9028 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:13:07.122807 9028 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260805 23:13:07.125059 9045 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:13:07.126439 9028 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:13:07.126585 9028 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-1/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-1/wal
uuid: "3f3cc8ff1e664a859a891b80666307f1"
format_stamp: "Formatted at 2026-08-05 23:13:04 on dist-test-slave-ss4n"
server_key: "0831a59f62202ce72c947f5a4bdb6bc4"
server_key_iv: "803573aa0d560547b97631878b70eb79"
server_key_version: "encryptionkey@0"
I20260805 23:13:07.127200 9028 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:13:07.145824 9028 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:13:07.149451 9028 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:13:07.149737 9028 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:13:07.150439 9028 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:13:07.151409 9028 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:13:07.151497 9028 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:13:07.151599 9028 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:13:07.151656 9028 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:13:07.162478 9028 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.194:37313
I20260805 23:13:07.162515 9163 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.194:37313 every 8 connection(s)
I20260805 23:13:07.163635 9028 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-1/data/info.pb
I20260805 23:13:07.164988 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 9028
I20260805 23:13:07.165445 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.92.195:38153
--local_ip_for_outbound_sockets=127.25.92.195
--tserver_master_addrs=127.25.92.254:45961
--webserver_port=44611
--webserver_interface=127.25.92.195
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.195.keytab
--principal=kudu/127.25.92.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:41983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260805 23:13:07.168167 9166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:07.166138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:45961 (local address 127.25.92.194:58417)
0805 23:13:07.166696 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0805 23:13:07.166870 (+ 174us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:45961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":113,"threads_started":1}
W20260805 23:13:07.168735 9164 heartbeater.cc:644] Failed to heartbeat to 127.25.92.254:45961 (0 consecutive failures): Network error: Failed to ping master at 127.25.92.254:45961: Client connection negotiation failed: client connection to 127.25.92.254:45961: connect: Connection refused (error 111)
I20260805 23:13:07.169454 9166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:07.169216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:45961 (local address 127.25.92.194:52279)
0805 23:13:07.169345 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0805 23:13:07.169397 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:45961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260805 23:13:07.170248 9166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:07.170004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:45961 (local address 127.25.92.194:42507)
0805 23:13:07.170141 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0805 23:13:07.170192 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:45961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260805 23:13:07.170478 9164 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260805 23:13:07.282348 9167 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:13:07.282732 9167 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:13:07.282778 9167 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:13:07.287240 9167 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:13:07.287318 9167 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:13:07.287393 9167 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.92.195
I20260805 23:13:07.292914 9167 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:41983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.195.keytab
--principal=kudu/127.25.92.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.92.195:38153
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.92.195
--webserver_port=44611
--tserver_master_addrs=127.25.92.254:45961
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9167
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.92.195
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:13:07.294113 9167 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:13:07.295205 9167 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:13:07.305343 9172 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:13:07.305343 9173 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:13:07.305362 9175 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:13:07.305853 9167 server_base.cc:1034] running on GCE node
I20260805 23:13:07.306447 9167 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:13:07.307257 9167 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:13:07.308493 9167 hybrid_clock.cc:648] HybridClock initialized: now 1785971587308451 us; error 55 us; skew 500 ppm
Aug 05 23:13:07 dist-test-slave-ss4n krb5kdc[7668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971587, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:13:07.311698 9167 init.cc:377] Logged in from keytab as kudu/127.25.92.195@KRBTEST.COM (short username kudu)
I20260805 23:13:07.312691 9167 webserver.cc:460] Webserver started at http://127.25.92.195:44611/ using document root <none> and password file <none>
I20260805 23:13:07.313354 9167 fs_manager.cc:359] Metadata directory not provided
I20260805 23:13:07.313458 9167 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:13:07.317850 9167 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:13:07.320370 9184 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:13:07.321955 9167 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:13:07.322121 9167 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-2/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-2/wal
uuid: "57ca592f72ed4d6f8fd9a07643f02fd5"
format_stamp: "Formatted at 2026-08-05 23:13:04 on dist-test-slave-ss4n"
server_key: "6148b95a70b48af2769e9dff78e44ce4"
server_key_iv: "27d0d2bb33804600190f62945429256a"
server_key_version: "encryptionkey@0"
I20260805 23:13:07.322791 9167 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:13:07.340737 9167 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:13:07.344580 9167 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:13:07.344837 9167 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:13:07.345580 9167 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:13:07.346508 9167 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:13:07.346591 9167 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:13:07.346670 9167 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:13:07.346724 9167 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:13:07.357577 9167 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.195:38153
I20260805 23:13:07.357625 9301 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.195:38153 every 8 connection(s)
I20260805 23:13:07.358698 9167 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971406251336-25971-0/minicluster-data/ts-2/data/info.pb
I20260805 23:13:07.363143 9304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:07.361115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:45961 (local address 127.25.92.195:45871)
0805 23:13:07.361697 (+ 582us) negotiation.cc:107] Waiting for socket to connect
0805 23:13:07.361826 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:45961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":139,"threads_started":1}
W20260805 23:13:07.363658 9302 heartbeater.cc:644] Failed to heartbeat to 127.25.92.254:45961 (0 consecutive failures): Network error: Failed to ping master at 127.25.92.254:45961: Client connection negotiation failed: client connection to 127.25.92.254:45961: connect: Connection refused (error 111)
I20260805 23:13:07.364197 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 9167
I20260805 23:13:07.364462 9304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:07.364187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:45961 (local address 127.25.92.195:54653)
0805 23:13:07.364328 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0805 23:13:07.364402 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:45961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260805 23:13:07.365247 9304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:07.364989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:45961 (local address 127.25.92.195:37687)
0805 23:13:07.365132 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0805 23:13:07.365192 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:45961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260805 23:13:07.365417 9302 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260805 23:13:07.997888 9306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:07.997339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:45961 (local address 127.25.92.193:45663)
0805 23:13:07.997722 (+ 383us) negotiation.cc:107] Waiting for socket to connect
0805 23:13:07.997785 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:45961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":83,"threads_started":1}
I20260805 23:13:08.171753 9307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:08.171108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:45961 (local address 127.25.92.194:43743)
0805 23:13:08.171388 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0805 23:13:08.171448 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:45961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":87,"threads_started":1}
I20260805 23:13:08.285315 9308 raft_consensus.cc:482] T 33897e3f589e4555a289d1715f962f90 P 9d7053e2a49447dc8f347c9188078e94 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 23:13:08.285564 9308 raft_consensus.cc:504] T 33897e3f589e4555a289d1715f962f90 P 9d7053e2a49447dc8f347c9188078e94 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57ca592f72ed4d6f8fd9a07643f02fd5" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38153 } } peers { permanent_uuid: "3f3cc8ff1e664a859a891b80666307f1" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 37313 } } peers { permanent_uuid: "9d7053e2a49447dc8f347c9188078e94" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 43069 } }
I20260805 23:13:08.286831 9308 leader_election.cc:288] T 33897e3f589e4555a289d1715f962f90 P 9d7053e2a49447dc8f347c9188078e94 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 57ca592f72ed4d6f8fd9a07643f02fd5 (127.25.92.195:38153), 3f3cc8ff1e664a859a891b80666307f1 (127.25.92.194:37313)
Aug 05 23:13:08 dist-test-slave-ss4n krb5kdc[7668](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971586, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.193@KRBTEST.COM for kudu/127.25.92.194@KRBTEST.COM
Aug 05 23:13:08 dist-test-slave-ss4n krb5kdc[7668](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971586, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.193@KRBTEST.COM for kudu/127.25.92.195@KRBTEST.COM
I20260805 23:13:08.305694 9311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:08.287332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.193:56857 (local address 127.25.92.194:37313)
0805 23:13:08.287681 (+ 349us) server_negotiation.cc:206] Beginning negotiation
0805 23:13:08.287710 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0805 23:13:08.287754 (+ 44us) server_negotiation.cc:407] Connection header received
0805 23:13:08.288640 (+ 886us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:13:08.288647 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:13:08.289086 (+ 439us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:13:08.289423 (+ 337us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:13:08.291380 (+ 1957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:13:08.292980 (+ 1600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:13:08.295296 (+ 2316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:13:08.296087 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:13:08.296282 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:13:08.300110 (+ 3828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:13:08.300124 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:13:08.300139 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:13:08.300174 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:13:08.302618 (+ 2444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:13:08.303401 (+ 783us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:13:08.303406 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:13:08.303410 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:13:08.303542 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:13:08.303976 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:13:08.303979 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:13:08.303980 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:13:08.304408 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:13:08.304556 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0805 23:13:08.305145 (+ 589us) server_negotiation.cc:299] Negotiation successful
0805 23:13:08.305441 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202,"thread_start_us":100,"threads_started":1}
I20260805 23:13:08.305689 9309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:08.287163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.194:37313 (local address 127.25.92.193:56857)
0805 23:13:08.287503 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0805 23:13:08.287517 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0805 23:13:08.287678 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:13:08.290017 (+ 2339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:13:08.290020 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:13:08.290330 (+ 310us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:13:08.291149 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:13:08.291172 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:13:08.293194 (+ 2022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:13:08.293198 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:13:08.295124 (+ 1926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:13:08.295138 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:13:08.296277 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:13:08.296282 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:13:08.296606 (+ 324us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:13:08.297569 (+ 963us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:13:08.297602 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:13:08.299797 (+ 2195us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:13:08.302802 (+ 3005us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:13:08.302809 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:13:08.302825 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:13:08.303233 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:13:08.303724 (+ 491us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:13:08.303727 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:13:08.303728 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:13:08.303836 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:13:08.304537 (+ 701us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:13:08.304544 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:13:08.304813 (+ 269us) client_negotiation.cc:769] Sending connection context
0805 23:13:08.305012 (+ 199us) client_negotiation.cc:240] Negotiation successful
0805 23:13:08.305464 (+ 452us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"mutex_wait_us":549,"spinlock_wait_cycles":40576,"thread_start_us":70,"threads_started":1}
I20260805 23:13:08.305688 9310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:08.287271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.193:49331 (local address 127.25.92.195:38153)
0805 23:13:08.287701 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0805 23:13:08.287712 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0805 23:13:08.287754 (+ 42us) server_negotiation.cc:407] Connection header received
0805 23:13:08.288621 (+ 867us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:13:08.288631 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:13:08.289086 (+ 455us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:13:08.289423 (+ 337us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:13:08.291380 (+ 1957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:13:08.292989 (+ 1609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:13:08.295296 (+ 2307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:13:08.296100 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:13:08.296281 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:13:08.300207 (+ 3926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:13:08.300212 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:13:08.300223 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:13:08.300255 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:13:08.302652 (+ 2397us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:13:08.303388 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:13:08.303395 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:13:08.303399 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:13:08.303541 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:13:08.303935 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:13:08.303942 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:13:08.303944 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:13:08.304410 (+ 466us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:13:08.304577 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0805 23:13:08.305151 (+ 574us) server_negotiation.cc:299] Negotiation successful
0805 23:13:08.305450 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":95,"threads_started":1}
I20260805 23:13:08.305738 9306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:08.287042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.195:38153 (local address 127.25.92.193:49331)
0805 23:13:08.287246 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0805 23:13:08.287262 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0805 23:13:08.287376 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:13:08.289680 (+ 2304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:13:08.289687 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:13:08.290330 (+ 643us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:13:08.291150 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:13:08.291172 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:13:08.293194 (+ 2022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:13:08.293198 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:13:08.295124 (+ 1926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:13:08.295138 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:13:08.296277 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:13:08.296282 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:13:08.296655 (+ 373us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:13:08.297569 (+ 914us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:13:08.297602 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:13:08.299983 (+ 2381us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:13:08.302802 (+ 2819us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:13:08.302823 (+ 21us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:13:08.302828 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:13:08.303227 (+ 399us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:13:08.303652 (+ 425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:13:08.303655 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:13:08.303658 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:13:08.303818 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:13:08.304555 (+ 737us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:13:08.304558 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:13:08.304813 (+ 255us) client_negotiation.cc:769] Sending connection context
0805 23:13:08.305014 (+ 201us) client_negotiation.cc:240] Negotiation successful
0805 23:13:08.305582 (+ 568us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"mutex_wait_us":669,"spinlock_wait_cycles":16896}
W20260805 23:13:08.308487 8906 leader_election.cc:341] T 33897e3f589e4555a289d1715f962f90 P 9d7053e2a49447dc8f347c9188078e94 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3f3cc8ff1e664a859a891b80666307f1 (127.25.92.194:37313): Not found: Tablet not found: 33897e3f589e4555a289d1715f962f90
W20260805 23:13:08.309191 8905 leader_election.cc:341] T 33897e3f589e4555a289d1715f962f90 P 9d7053e2a49447dc8f347c9188078e94 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 57ca592f72ed4d6f8fd9a07643f02fd5 (127.25.92.195:38153): Not found: Tablet not found: 33897e3f589e4555a289d1715f962f90
I20260805 23:13:08.309281 8905 leader_election.cc:302] T 33897e3f589e4555a289d1715f962f90 P 9d7053e2a49447dc8f347c9188078e94 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9d7053e2a49447dc8f347c9188078e94; no voters: 3f3cc8ff1e664a859a891b80666307f1, 57ca592f72ed4d6f8fd9a07643f02fd5
I20260805 23:13:08.309672 9308 raft_consensus.cc:2726] T 33897e3f589e4555a289d1715f962f90 P 9d7053e2a49447dc8f347c9188078e94 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260805 23:13:08.366590 9312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:08.366161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:45961 (local address 127.25.92.195:58985)
0805 23:13:08.366423 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0805 23:13:08.366486 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:45961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":76,"threads_started":1}
I20260805 23:13:08.999269 9313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:08.998786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:45961 (local address 127.25.92.193:38385)
0805 23:13:08.999094 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0805 23:13:08.999167 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:45961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":85,"threads_started":1}
I20260805 23:13:09.173111 9314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:09.172700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:45961 (local address 127.25.92.194:35735)
0805 23:13:09.172957 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0805 23:13:09.173019 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:45961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":78,"threads_started":1}
I20260805 23:13:09.367945 9315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:09.367429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:45961 (local address 127.25.92.195:51795)
0805 23:13:09.367755 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0805 23:13:09.367826 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:45961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":99,"threads_started":1}
I20260805 23:13:10.000706 9316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:10.000221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:45961 (local address 127.25.92.193:56689)
0805 23:13:10.000538 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0805 23:13:10.000607 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:45961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":100,"threads_started":1}
I20260805 23:13:10.067742 9317 raft_consensus.cc:482] T 33897e3f589e4555a289d1715f962f90 P 9d7053e2a49447dc8f347c9188078e94 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 23:13:10.067915 9317 raft_consensus.cc:504] T 33897e3f589e4555a289d1715f962f90 P 9d7053e2a49447dc8f347c9188078e94 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57ca592f72ed4d6f8fd9a07643f02fd5" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38153 } } peers { permanent_uuid: "3f3cc8ff1e664a859a891b80666307f1" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 37313 } } peers { permanent_uuid: "9d7053e2a49447dc8f347c9188078e94" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 43069 } }
I20260805 23:13:10.068602 9317 leader_election.cc:288] T 33897e3f589e4555a289d1715f962f90 P 9d7053e2a49447dc8f347c9188078e94 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 57ca592f72ed4d6f8fd9a07643f02fd5 (127.25.92.195:38153), 3f3cc8ff1e664a859a891b80666307f1 (127.25.92.194:37313)
W20260805 23:13:10.069626 8906 leader_election.cc:341] T 33897e3f589e4555a289d1715f962f90 P 9d7053e2a49447dc8f347c9188078e94 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3f3cc8ff1e664a859a891b80666307f1 (127.25.92.194:37313): Not found: Tablet not found: 33897e3f589e4555a289d1715f962f90
W20260805 23:13:10.069805 8905 leader_election.cc:341] T 33897e3f589e4555a289d1715f962f90 P 9d7053e2a49447dc8f347c9188078e94 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 57ca592f72ed4d6f8fd9a07643f02fd5 (127.25.92.195:38153): Not found: Tablet not found: 33897e3f589e4555a289d1715f962f90
I20260805 23:13:10.069921 8905 leader_election.cc:302] T 33897e3f589e4555a289d1715f962f90 P 9d7053e2a49447dc8f347c9188078e94 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9d7053e2a49447dc8f347c9188078e94; no voters: 3f3cc8ff1e664a859a891b80666307f1, 57ca592f72ed4d6f8fd9a07643f02fd5
I20260805 23:13:10.070068 9317 raft_consensus.cc:2726] T 33897e3f589e4555a289d1715f962f90 P 9d7053e2a49447dc8f347c9188078e94 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260805 23:13:10.174573 9318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:10.174079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:45961 (local address 127.25.92.194:52109)
0805 23:13:10.174402 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0805 23:13:10.174466 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:45961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":112,"threads_started":1}
I20260805 23:13:10.369395 9319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:10.368889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:45961 (local address 127.25.92.195:53877)
0805 23:13:10.369205 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0805 23:13:10.369272 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:45961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":95,"threads_started":1}
I20260805 23:13:11.002159 9320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:11.001709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:45961 (local address 127.25.92.193:55763)
0805 23:13:11.001995 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0805 23:13:11.002056 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:45961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":84,"threads_started":1}
I20260805 23:13:11.176051 9321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:11.175551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:45961 (local address 127.25.92.194:43475)
0805 23:13:11.175865 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0805 23:13:11.175941 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:45961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":94,"threads_started":1}
I20260805 23:13:11.370781 9322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:11.370336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:45961 (local address 127.25.92.195:51439)
0805 23:13:11.370618 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0805 23:13:11.370684 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:45961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":85,"threads_started":1}
I20260805 23:13:12.003669 9323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:12.003171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:45961 (local address 127.25.92.193:59959)
0805 23:13:12.003474 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0805 23:13:12.003532 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:45961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":91,"threads_started":1}
I20260805 23:13:12.177753 9324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:12.177059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:45961 (local address 127.25.92.194:58893)
0805 23:13:12.177438 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0805 23:13:12.177618 (+ 180us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:45961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":159,"threads_started":1}
I20260805 23:13:12.372331 9325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:12.371856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:45961 (local address 127.25.92.195:41837)
0805 23:13:12.372169 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0805 23:13:12.372234 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:45961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":99,"threads_started":1}
I20260805 23:13:13.005025 9326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:13.004559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:45961 (local address 127.25.92.193:52945)
0805 23:13:13.004854 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0805 23:13:13.004919 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:45961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":84,"threads_started":1}
I20260805 23:13:13.179149 9327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:13.178654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:45961 (local address 127.25.92.194:49783)
0805 23:13:13.178945 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0805 23:13:13.179034 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:45961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":109,"threads_started":1}
I20260805 23:13:13.373881 9328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:13.373335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:45961 (local address 127.25.92.195:54013)
0805 23:13:13.373720 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0805 23:13:13.373782 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:45961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":101,"threads_started":1}
I20260805 23:13:13.438297 9329 raft_consensus.cc:482] T 33897e3f589e4555a289d1715f962f90 P 9d7053e2a49447dc8f347c9188078e94 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 23:13:13.438449 9329 raft_consensus.cc:504] T 33897e3f589e4555a289d1715f962f90 P 9d7053e2a49447dc8f347c9188078e94 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57ca592f72ed4d6f8fd9a07643f02fd5" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38153 } } peers { permanent_uuid: "3f3cc8ff1e664a859a891b80666307f1" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 37313 } } peers { permanent_uuid: "9d7053e2a49447dc8f347c9188078e94" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 43069 } }
I20260805 23:13:13.438982 9329 leader_election.cc:288] T 33897e3f589e4555a289d1715f962f90 P 9d7053e2a49447dc8f347c9188078e94 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 57ca592f72ed4d6f8fd9a07643f02fd5 (127.25.92.195:38153), 3f3cc8ff1e664a859a891b80666307f1 (127.25.92.194:37313)
W20260805 23:13:13.439828 8905 leader_election.cc:341] T 33897e3f589e4555a289d1715f962f90 P 9d7053e2a49447dc8f347c9188078e94 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 57ca592f72ed4d6f8fd9a07643f02fd5 (127.25.92.195:38153): Not found: Tablet not found: 33897e3f589e4555a289d1715f962f90
W20260805 23:13:13.439937 8906 leader_election.cc:341] T 33897e3f589e4555a289d1715f962f90 P 9d7053e2a49447dc8f347c9188078e94 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3f3cc8ff1e664a859a891b80666307f1 (127.25.92.194:37313): Not found: Tablet not found: 33897e3f589e4555a289d1715f962f90
I20260805 23:13:13.440056 8906 leader_election.cc:302] T 33897e3f589e4555a289d1715f962f90 P 9d7053e2a49447dc8f347c9188078e94 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9d7053e2a49447dc8f347c9188078e94; no voters: 3f3cc8ff1e664a859a891b80666307f1, 57ca592f72ed4d6f8fd9a07643f02fd5
I20260805 23:13:13.440232 9329 raft_consensus.cc:2726] T 33897e3f589e4555a289d1715f962f90 P 9d7053e2a49447dc8f347c9188078e94 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260805 23:13:14.006390 9330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:14.005937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:45961 (local address 127.25.92.193:53341)
0805 23:13:14.006226 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0805 23:13:14.006291 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:45961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":82,"threads_started":1}
I20260805 23:13:14.180397 9331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:14.179892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:45961 (local address 127.25.92.194:54915)
0805 23:13:14.180232 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0805 23:13:14.180298 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:45961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":92,"threads_started":1}
I20260805 23:13:14.375572 9332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:14.375006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:45961 (local address 127.25.92.195:41483)
0805 23:13:14.375388 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0805 23:13:14.375475 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:45961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":100,"threads_started":1}
I20260805 23:13:15.008195 9333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:15.007575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:45961 (local address 127.25.92.193:35533)
0805 23:13:15.007927 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0805 23:13:15.008078 (+ 151us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:45961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":130,"threads_started":1}
I20260805 23:13:15.181717 9334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:15.181182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:45961 (local address 127.25.92.194:33957)
0805 23:13:15.181464 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0805 23:13:15.181592 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:45961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":78,"threads_started":1}
I20260805 23:13:15.377010 9335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:15.376524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:45961 (local address 127.25.92.195:37359)
0805 23:13:15.376846 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0805 23:13:15.376911 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:45961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":103,"threads_started":1}
I20260805 23:13:16.009655 9336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:16.009112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:45961 (local address 127.25.92.193:33045)
0805 23:13:16.009413 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0805 23:13:16.009469 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:45961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":78,"threads_started":1}
I20260805 23:13:16.183279 9337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:16.182760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:45961 (local address 127.25.92.194:49971)
0805 23:13:16.183113 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0805 23:13:16.183182 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:45961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":102,"threads_started":1}
I20260805 23:13:16.378537 9338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:16.378080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:45961 (local address 127.25.92.195:35945)
0805 23:13:16.378377 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0805 23:13:16.378439 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:45961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":100,"threads_started":1}
I20260805 23:13:16.754788 9339 raft_consensus.cc:482] T 33897e3f589e4555a289d1715f962f90 P 9d7053e2a49447dc8f347c9188078e94 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 23:13:16.755002 9339 raft_consensus.cc:504] T 33897e3f589e4555a289d1715f962f90 P 9d7053e2a49447dc8f347c9188078e94 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57ca592f72ed4d6f8fd9a07643f02fd5" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38153 } } peers { permanent_uuid: "3f3cc8ff1e664a859a891b80666307f1" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 37313 } } peers { permanent_uuid: "9d7053e2a49447dc8f347c9188078e94" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 43069 } }
I20260805 23:13:16.755563 9339 leader_election.cc:288] T 33897e3f589e4555a289d1715f962f90 P 9d7053e2a49447dc8f347c9188078e94 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 57ca592f72ed4d6f8fd9a07643f02fd5 (127.25.92.195:38153), 3f3cc8ff1e664a859a891b80666307f1 (127.25.92.194:37313)
W20260805 23:13:16.756354 8905 leader_election.cc:341] T 33897e3f589e4555a289d1715f962f90 P 9d7053e2a49447dc8f347c9188078e94 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 57ca592f72ed4d6f8fd9a07643f02fd5 (127.25.92.195:38153): Not found: Tablet not found: 33897e3f589e4555a289d1715f962f90
W20260805 23:13:16.756467 8906 leader_election.cc:341] T 33897e3f589e4555a289d1715f962f90 P 9d7053e2a49447dc8f347c9188078e94 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3f3cc8ff1e664a859a891b80666307f1 (127.25.92.194:37313): Not found: Tablet not found: 33897e3f589e4555a289d1715f962f90
I20260805 23:13:16.756585 8906 leader_election.cc:302] T 33897e3f589e4555a289d1715f962f90 P 9d7053e2a49447dc8f347c9188078e94 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9d7053e2a49447dc8f347c9188078e94; no voters: 3f3cc8ff1e664a859a891b80666307f1, 57ca592f72ed4d6f8fd9a07643f02fd5
I20260805 23:13:16.756743 9339 raft_consensus.cc:2726] T 33897e3f589e4555a289d1715f962f90 P 9d7053e2a49447dc8f347c9188078e94 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260805 23:13:17.011097 9340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:17.010569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:45961 (local address 127.25.92.193:58049)
0805 23:13:17.010914 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0805 23:13:17.010976 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:45961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":119,"threads_started":1}
I20260805 23:13:17.184921 9341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:17.184277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:45961 (local address 127.25.92.194:44219)
0805 23:13:17.184711 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0805 23:13:17.184790 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:45961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":117,"threads_started":1}
I20260805 23:13:17.380087 9342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:17.379501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:45961 (local address 127.25.92.195:49695)
0805 23:13:17.379909 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0805 23:13:17.379985 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:45961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":165,"threads_started":1}
I20260805 23:13:18.012403 9343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:18.011980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:45961 (local address 127.25.92.193:49345)
0805 23:13:18.012243 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0805 23:13:18.012300 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:45961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":87,"threads_started":1}
I20260805 23:13:18.186373 9344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:18.185886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:45961 (local address 127.25.92.194:39947)
0805 23:13:18.186211 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0805 23:13:18.186275 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:45961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":107,"threads_started":1}
I20260805 23:13:18.381685 9345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:18.381151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:45961 (local address 127.25.92.195:41019)
0805 23:13:18.381439 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0805 23:13:18.381563 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:45961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":106,"threads_started":1}
I20260805 23:13:19.013891 9346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:19.013324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:45961 (local address 127.25.92.193:56807)
0805 23:13:19.013712 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0805 23:13:19.013792 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:45961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":85,"threads_started":1}
I20260805 23:13:19.187816 9347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:19.187298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:45961 (local address 127.25.92.194:43205)
0805 23:13:19.187652 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0805 23:13:19.187714 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:45961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":88,"threads_started":1}
I20260805 23:13:19.383034 9348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:19.382584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:45961 (local address 127.25.92.195:42571)
0805 23:13:19.382869 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0805 23:13:19.382934 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:45961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":97,"threads_started":1}
I20260805 23:13:20.015426 9349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:20.014879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:45961 (local address 127.25.92.193:34091)
0805 23:13:20.015246 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0805 23:13:20.015324 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:45961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":98,"threads_started":1}
I20260805 23:13:20.189284 9350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:20.188792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:45961 (local address 127.25.92.194:41333)
0805 23:13:20.189103 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0805 23:13:20.189174 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:45961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":79,"threads_started":1}
I20260805 23:13:20.385002 9351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:20.384353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:45961 (local address 127.25.92.195:46203)
0805 23:13:20.384789 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0805 23:13:20.384902 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:45961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":160,"threads_started":1}
I20260805 23:13:20.569399 9352 raft_consensus.cc:482] T 33897e3f589e4555a289d1715f962f90 P 9d7053e2a49447dc8f347c9188078e94 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 23:13:20.569579 9352 raft_consensus.cc:504] T 33897e3f589e4555a289d1715f962f90 P 9d7053e2a49447dc8f347c9188078e94 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57ca592f72ed4d6f8fd9a07643f02fd5" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 38153 } } peers { permanent_uuid: "3f3cc8ff1e664a859a891b80666307f1" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 37313 } } peers { permanent_uuid: "9d7053e2a49447dc8f347c9188078e94" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 43069 } }
I20260805 23:13:20.570063 9352 leader_election.cc:288] T 33897e3f589e4555a289d1715f962f90 P 9d7053e2a49447dc8f347c9188078e94 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 57ca592f72ed4d6f8fd9a07643f02fd5 (127.25.92.195:38153), 3f3cc8ff1e664a859a891b80666307f1 (127.25.92.194:37313)
W20260805 23:13:20.570878 8906 leader_election.cc:341] T 33897e3f589e4555a289d1715f962f90 P 9d7053e2a49447dc8f347c9188078e94 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3f3cc8ff1e664a859a891b80666307f1 (127.25.92.194:37313): Not found: Tablet not found: 33897e3f589e4555a289d1715f962f90
W20260805 23:13:20.570974 8905 leader_election.cc:341] T 33897e3f589e4555a289d1715f962f90 P 9d7053e2a49447dc8f347c9188078e94 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 57ca592f72ed4d6f8fd9a07643f02fd5 (127.25.92.195:38153): Not found: Tablet not found: 33897e3f589e4555a289d1715f962f90
I20260805 23:13:20.571107 8905 leader_election.cc:302] T 33897e3f589e4555a289d1715f962f90 P 9d7053e2a49447dc8f347c9188078e94 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9d7053e2a49447dc8f347c9188078e94; no voters: 3f3cc8ff1e664a859a891b80666307f1, 57ca592f72ed4d6f8fd9a07643f02fd5
I20260805 23:13:20.571326 9352 raft_consensus.cc:2726] T 33897e3f589e4555a289d1715f962f90 P 9d7053e2a49447dc8f347c9188078e94 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260805 23:13:21.017117 9353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:21.016574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:45961 (local address 127.25.92.193:44889)
0805 23:13:21.016942 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0805 23:13:21.017015 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:45961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":129,"threads_started":1}
I20260805 23:13:21.190899 9354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:21.190405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:45961 (local address 127.25.92.194:47563)
0805 23:13:21.190719 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0805 23:13:21.190789 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:45961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":84,"threads_started":1}
I20260805 23:13:21.386382 9355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:21.385901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:45961 (local address 127.25.92.195:59507)
0805 23:13:21.386214 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0805 23:13:21.386283 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:45961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":87,"threads_started":1}
I20260805 23:13:22.018880 9356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:22.018400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:45961 (local address 127.25.92.193:40107)
0805 23:13:22.018718 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0805 23:13:22.018781 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:45961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":97,"threads_started":1}
I20260805 23:13:22.192716 9357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:22.192130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:45961 (local address 127.25.92.194:40347)
0805 23:13:22.192472 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0805 23:13:22.192599 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:45961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":94,"threads_started":1}
I20260805 23:13:22.366209 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 8852
I20260805 23:13:22.372659 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 9028
I20260805 23:13:22.379202 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 9167
2026-08-05T23:13:22Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19842 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 23:13:22 dist-test-slave-ss4n krb5kdc[9361](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 23:13:22 dist-test-slave-ss4n krb5kdc[9361](info): set up 2 sockets
Aug 05 23:13:22 dist-test-slave-ss4n krb5kdc[9361](info): commencing operation
krb5kdc: starting...
W20260805 23:13:23.448321 25971 mini_kdc.cc:121] Time spent starting KDC: real 1.044s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/krb5kdc/test-user.keytab.
Aug 05 23:13:23 dist-test-slave-ss4n krb5kdc[9361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971603, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T23:13:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:13:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.92.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.92.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.92.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab.
Entry for principal kudu/127.25.92.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab.
I20260805 23:13:23.586521 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.254:34379
--webserver_interface=127.25.92.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab
--principal=kudu/127.25.92.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:33407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.92.254:34379
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:13:23.699496 9374 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:13:23.699874 9374 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:13:23.699954 9374 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:13:23.704243 9374 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260805 23:13:23.704360 9374 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260805 23:13:23.704389 9374 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260805 23:13:23.704412 9374 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:13:23.704456 9374 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:13:23.704494 9374 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:13:23.704553 9374 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:13:23.704596 9374 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:13:23.710438 9374 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:33407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.25.92.254:34379
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab
--principal=kudu/127.25.92.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.254:34379
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.92.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9374
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:13:23.711678 9374 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:13:23.712782 9374 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:13:23.722335 9382 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:13:23.722476 9380 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:13:23.722433 9374 server_base.cc:1034] running on GCE node
W20260805 23:13:23.722339 9379 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:13:23.723348 9374 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:13:23.724674 9374 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:13:23.725862 9374 hybrid_clock.cc:648] HybridClock initialized: now 1785971603725839 us; error 37 us; skew 500 ppm
Aug 05 23:13:23 dist-test-slave-ss4n krb5kdc[9361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971603, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:13:23.729306 9374 init.cc:377] Logged in from keytab as kudu/127.25.92.254@KRBTEST.COM (short username kudu)
I20260805 23:13:23.730531 9374 webserver.cc:460] Webserver started at http://127.25.92.254:40893/ using document root <none> and password file <none>
I20260805 23:13:23.731106 9374 fs_manager.cc:359] Metadata directory not provided
I20260805 23:13:23.731169 9374 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:13:23.731359 9374 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:13:23.733564 9374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/master-0/data/instance:
uuid: "242b9070a63b4c049d7c7563d64f021b"
format_stamp: "Formatted at 2026-08-05 23:13:23 on dist-test-slave-ss4n"
server_key: "9cde2619d4310f38000f1ef33c06a78a"
server_key_iv: "f6299010fe621bb41c17e89177c7cdaa"
server_key_version: "encryptionkey@0"
I20260805 23:13:23.734130 9374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/master-0/wal/instance:
uuid: "242b9070a63b4c049d7c7563d64f021b"
format_stamp: "Formatted at 2026-08-05 23:13:23 on dist-test-slave-ss4n"
server_key: "9cde2619d4310f38000f1ef33c06a78a"
server_key_iv: "f6299010fe621bb41c17e89177c7cdaa"
server_key_version: "encryptionkey@0"
I20260805 23:13:23.738090 9374 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.003s
I20260805 23:13:23.740706 9389 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:13:23.742208 9374 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:13:23.742357 9374 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/master-0/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/master-0/wal
uuid: "242b9070a63b4c049d7c7563d64f021b"
format_stamp: "Formatted at 2026-08-05 23:13:23 on dist-test-slave-ss4n"
server_key: "9cde2619d4310f38000f1ef33c06a78a"
server_key_iv: "f6299010fe621bb41c17e89177c7cdaa"
server_key_version: "encryptionkey@0"
I20260805 23:13:23.742578 9374 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:13:23.773590 9374 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:13:23.777201 9374 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:13:23.777417 9374 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:13:23.784945 9374 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.254:34379
I20260805 23:13:23.785328 9445 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.254:34379 every 8 connection(s)
I20260805 23:13:23.785993 9374 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
I20260805 23:13:23.788959 9446 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:13:23.794476 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 9374
I20260805 23:13:23.794586 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/master-0/wal/instance
I20260805 23:13:23.794885 25971 external_mini_cluster.cc:1428] Setting key b6f40c33fe1b25122a2534d9162c8da0
I20260805 23:13:23.795142 9446 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 242b9070a63b4c049d7c7563d64f021b: Bootstrap starting.
I20260805 23:13:23.798050 9446 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 242b9070a63b4c049d7c7563d64f021b: Neither blocks nor log segments found. Creating new log.
I20260805 23:13:23.798908 9446 log.cc:824] T 00000000000000000000000000000000 P 242b9070a63b4c049d7c7563d64f021b: Log is configured to *not* fsync() on all Append() calls
I20260805 23:13:23.801054 9446 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 242b9070a63b4c049d7c7563d64f021b: No bootstrap required, opened a new log
Aug 05 23:13:23 dist-test-slave-ss4n krb5kdc[9361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971603, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM
I20260805 23:13:23.806321 9446 raft_consensus.cc:348] T 00000000000000000000000000000000 P 242b9070a63b4c049d7c7563d64f021b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "242b9070a63b4c049d7c7563d64f021b" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 34379 } }
I20260805 23:13:23.806644 9446 raft_consensus.cc:374] T 00000000000000000000000000000000 P 242b9070a63b4c049d7c7563d64f021b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:13:23.806689 9446 raft_consensus.cc:729] T 00000000000000000000000000000000 P 242b9070a63b4c049d7c7563d64f021b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242b9070a63b4c049d7c7563d64f021b, State: Initialized, Role: FOLLOWER
I20260805 23:13:23.807219 9446 consensus_queue.cc:260] T 00000000000000000000000000000000 P 242b9070a63b4c049d7c7563d64f021b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "242b9070a63b4c049d7c7563d64f021b" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 34379 } }
I20260805 23:13:23.807406 9446 raft_consensus.cc:388] T 00000000000000000000000000000000 P 242b9070a63b4c049d7c7563d64f021b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:13:23.807502 9446 raft_consensus.cc:482] T 00000000000000000000000000000000 P 242b9070a63b4c049d7c7563d64f021b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:13:23.807632 9446 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 242b9070a63b4c049d7c7563d64f021b [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:13:23.808887 9446 raft_consensus.cc:504] T 00000000000000000000000000000000 P 242b9070a63b4c049d7c7563d64f021b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "242b9070a63b4c049d7c7563d64f021b" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 34379 } }
I20260805 23:13:23.809356 9446 leader_election.cc:302] T 00000000000000000000000000000000 P 242b9070a63b4c049d7c7563d64f021b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 242b9070a63b4c049d7c7563d64f021b; no voters:
I20260805 23:13:23.809880 9446 leader_election.cc:288] T 00000000000000000000000000000000 P 242b9070a63b4c049d7c7563d64f021b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 23:13:23.809980 9451 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 242b9070a63b4c049d7c7563d64f021b [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:13:23.810985 9449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:23.796375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44544 (local address 127.25.92.254:34379)
0805 23:13:23.796802 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0805 23:13:23.796834 (+ 32us) server_negotiation.cc:399] Waiting for connection header
0805 23:13:23.796871 (+ 37us) server_negotiation.cc:407] Connection header received
0805 23:13:23.797608 (+ 737us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:13:23.797627 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:13:23.797957 (+ 330us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:13:23.798309 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:13:23.799122 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:13:23.800104 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:13:23.800992 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:13:23.801638 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:13:23.801817 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:13:23.805047 (+ 3230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:13:23.805052 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:13:23.805069 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:13:23.805103 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:13:23.807233 (+ 2130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:13:23.807823 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:13:23.807828 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:13:23.807832 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:13:23.807950 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:13:23.808321 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:13:23.808324 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:13:23.808325 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:13:23.808856 (+ 531us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:13:23.809056 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0805 23:13:23.809404 (+ 348us) server_negotiation.cc:299] Negotiation successful
0805 23:13:23.809856 (+ 452us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":147,"threads_started":1}
I20260805 23:13:23.811367 9446 sys_catalog.cc:564] T 00000000000000000000000000000000 P 242b9070a63b4c049d7c7563d64f021b [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:13:23.810215 9451 raft_consensus.cc:686] T 00000000000000000000000000000000 P 242b9070a63b4c049d7c7563d64f021b [term 1 LEADER]: Becoming Leader. State: Replica: 242b9070a63b4c049d7c7563d64f021b, State: Running, Role: LEADER
I20260805 23:13:23.812472 9451 consensus_queue.cc:237] T 00000000000000000000000000000000 P 242b9070a63b4c049d7c7563d64f021b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "242b9070a63b4c049d7c7563d64f021b" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 34379 } }
I20260805 23:13:23.814406 9452 sys_catalog.cc:455] T 00000000000000000000000000000000 P 242b9070a63b4c049d7c7563d64f021b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "242b9070a63b4c049d7c7563d64f021b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "242b9070a63b4c049d7c7563d64f021b" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 34379 } } }
I20260805 23:13:23.814430 9453 sys_catalog.cc:455] T 00000000000000000000000000000000 P 242b9070a63b4c049d7c7563d64f021b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 242b9070a63b4c049d7c7563d64f021b. Latest consensus state: current_term: 1 leader_uuid: "242b9070a63b4c049d7c7563d64f021b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "242b9070a63b4c049d7c7563d64f021b" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 34379 } } }
I20260805 23:13:23.814570 9453 sys_catalog.cc:458] T 00000000000000000000000000000000 P 242b9070a63b4c049d7c7563d64f021b [sys.catalog]: This master's current role is: LEADER
I20260805 23:13:23.814570 9452 sys_catalog.cc:458] T 00000000000000000000000000000000 P 242b9070a63b4c049d7c7563d64f021b [sys.catalog]: This master's current role is: LEADER
W20260805 23:13:23.817564 9465 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 242b9070a63b4c049d7c7563d64f021b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260805 23:13:23.817662 9465 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260805 23:13:23.817755 9466 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:13:23.818776 9466 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:13:23.825212 9466 catalog_manager.cc:1347] Generated new cluster ID: bd8331ed9dd64faca326013e7f711783
I20260805 23:13:23.825325 9466 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:13:23.832579 9466 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:13:23.833949 9466 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:13:23.843021 9466 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 242b9070a63b4c049d7c7563d64f021b: Generated new TSK 0
I20260805 23:13:23.843947 9466 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.92.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.92.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.92.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.193.keytab.
Entry for principal kudu/127.25.92.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.193.keytab.
I20260805 23:13:23.877928 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.92.193:0
--local_ip_for_outbound_sockets=127.25.92.193
--webserver_interface=127.25.92.193
--webserver_port=0
--tserver_master_addrs=127.25.92.254:34379
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.193.keytab
--principal=kudu/127.25.92.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:33407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:13:24.003342 9472 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:13:24.003815 9472 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:13:24.003870 9472 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:13:24.008363 9472 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:13:24.008436 9472 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:13:24.008531 9472 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.92.193
I20260805 23:13:24.014428 9472 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:33407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.193.keytab
--principal=kudu/127.25.92.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.92.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.92.193
--webserver_port=0
--tserver_master_addrs=127.25.92.254:34379
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9472
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.92.193
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:13:24.015728 9472 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:13:24.016825 9472 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:13:24.027391 9477 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:13:24.027386 9480 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:13:24.027386 9478 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:13:24.027431 9472 server_base.cc:1034] running on GCE node
I20260805 23:13:24.028077 9472 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:13:24.028818 9472 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:13:24.030019 9472 hybrid_clock.cc:648] HybridClock initialized: now 1785971604029999 us; error 35 us; skew 500 ppm
Aug 05 23:13:24 dist-test-slave-ss4n krb5kdc[9361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971604, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:13:24.033265 9472 init.cc:377] Logged in from keytab as kudu/127.25.92.193@KRBTEST.COM (short username kudu)
I20260805 23:13:24.034389 9472 webserver.cc:460] Webserver started at http://127.25.92.193:43515/ using document root <none> and password file <none>
I20260805 23:13:24.035014 9472 fs_manager.cc:359] Metadata directory not provided
I20260805 23:13:24.035082 9472 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:13:24.035256 9472 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:13:24.037546 9472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-0/data/instance:
uuid: "85d51d37bc304498873dd204279742da"
format_stamp: "Formatted at 2026-08-05 23:13:24 on dist-test-slave-ss4n"
server_key: "b2eecfd72b6f80b9d7e2d72f01a973d8"
server_key_iv: "f4d7aa860e2f69542f8b62006293d9a9"
server_key_version: "encryptionkey@0"
I20260805 23:13:24.038133 9472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-0/wal/instance:
uuid: "85d51d37bc304498873dd204279742da"
format_stamp: "Formatted at 2026-08-05 23:13:24 on dist-test-slave-ss4n"
server_key: "b2eecfd72b6f80b9d7e2d72f01a973d8"
server_key_iv: "f4d7aa860e2f69542f8b62006293d9a9"
server_key_version: "encryptionkey@0"
I20260805 23:13:24.042323 9472 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260805 23:13:24.045297 9487 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:13:24.046891 9472 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260805 23:13:24.047037 9472 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-0/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-0/wal
uuid: "85d51d37bc304498873dd204279742da"
format_stamp: "Formatted at 2026-08-05 23:13:24 on dist-test-slave-ss4n"
server_key: "b2eecfd72b6f80b9d7e2d72f01a973d8"
server_key_iv: "f4d7aa860e2f69542f8b62006293d9a9"
server_key_version: "encryptionkey@0"
I20260805 23:13:24.047179 9472 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:13:24.070647 9472 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:13:24.074285 9472 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:13:24.074512 9472 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:13:24.075183 9472 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:13:24.076252 9472 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:13:24.076323 9472 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:13:24.076393 9472 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:13:24.076444 9472 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:13:24.085845 9472 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.193:45305
I20260805 23:13:24.085862 9604 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.193:45305 every 8 connection(s)
I20260805 23:13:24.086853 9472 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-0/data/info.pb
I20260805 23:13:24.096692 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 9472
I20260805 23:13:24.096843 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-0/wal/instance
Aug 05 23:13:24 dist-test-slave-ss4n krb5kdc[9361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971604, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.193@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM
I20260805 23:13:24.097178 25971 external_mini_cluster.cc:1428] Setting key 98c4e5fd0145aa93fdc8fd052b8359f2
I20260805 23:13:24.102901 9449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:24.088710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.193:35741 (local address 127.25.92.254:34379)
0805 23:13:24.088870 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0805 23:13:24.088874 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:13:24.089806 (+ 932us) server_negotiation.cc:407] Connection header received
0805 23:13:24.091222 (+ 1416us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:13:24.091225 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:13:24.091288 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:13:24.091367 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:13:24.092594 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:13:24.093158 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:13:24.094379 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:13:24.094626 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:13:24.094747 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:13:24.097918 (+ 3171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:13:24.097922 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:13:24.097925 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:13:24.097990 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:13:24.099984 (+ 1994us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:13:24.100596 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:13:24.100602 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:13:24.100607 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:13:24.100711 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:13:24.101341 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:13:24.101344 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:13:24.101345 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:13:24.101715 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:13:24.101814 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0805 23:13:24.102646 (+ 832us) server_negotiation.cc:299] Negotiation successful
0805 23:13:24.102771 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260805 23:13:24.104321 9607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:24.089102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:34379 (local address 127.25.92.193:35741)
0805 23:13:24.089636 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0805 23:13:24.089682 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0805 23:13:24.090994 (+ 1312us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:13:24.091551 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:13:24.091558 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:13:24.091929 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:13:24.092423 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:13:24.092437 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:13:24.093303 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:13:24.093309 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:13:24.094226 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:13:24.094237 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:13:24.094797 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:13:24.094800 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:13:24.094982 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:13:24.095666 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:13:24.095684 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:13:24.097651 (+ 1967us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:13:24.100150 (+ 2499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:13:24.100156 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:13:24.100171 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:13:24.100474 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:13:24.100857 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:13:24.100864 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:13:24.100867 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:13:24.101134 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:13:24.101848 (+ 714us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:13:24.101856 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:13:24.102292 (+ 436us) client_negotiation.cc:769] Sending connection context
0805 23:13:24.102539 (+ 247us) client_negotiation.cc:240] Negotiation successful
0805 23:13:24.102984 (+ 445us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":106,"threads_started":1}
I20260805 23:13:24.106096 9605 heartbeater.cc:347] Connected to a master server at 127.25.92.254:34379
I20260805 23:13:24.106418 9605 heartbeater.cc:464] Registering TS with master...
I20260805 23:13:24.107048 9605 heartbeater.cc:511] Master 127.25.92.254:34379 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.25.92.194@KRBTEST.COM; defaulting to no policy
I20260805 23:13:24.108764 9408 ts_manager.cc:194] Registered new tserver with Master: 85d51d37bc304498873dd204279742da (127.25.92.193:45305)
I20260805 23:13:24.110212 9408 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.92.193@KRBTEST.COM'} at 127.25.92.193:35741
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.92.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.92.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.194.keytab.
Entry for principal kudu/127.25.92.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.194.keytab.
I20260805 23:13:24.129663 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.92.194:0
--local_ip_for_outbound_sockets=127.25.92.194
--webserver_interface=127.25.92.194
--webserver_port=0
--tserver_master_addrs=127.25.92.254:34379
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.194.keytab
--principal=kudu/127.25.92.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:33407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:13:24.243371 9610 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:13:24.243664 9610 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:13:24.243709 9610 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:13:24.248001 9610 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:13:24.248075 9610 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:13:24.248159 9610 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.92.194
I20260805 23:13:24.253952 9610 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:33407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.194.keytab
--principal=kudu/127.25.92.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.92.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.92.194
--webserver_port=0
--tserver_master_addrs=127.25.92.254:34379
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9610
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.92.194
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:13:24.255177 9610 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:13:24.256249 9610 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:13:24.266116 9618 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:13:24.266099 9615 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:13:24.266085 9616 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:13:24.266287 9610 server_base.cc:1034] running on GCE node
I20260805 23:13:24.266932 9610 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:13:24.267671 9610 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:13:24.268889 9610 hybrid_clock.cc:648] HybridClock initialized: now 1785971604268866 us; error 42 us; skew 500 ppm
Aug 05 23:13:24 dist-test-slave-ss4n krb5kdc[9361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971604, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:13:24.272120 9610 init.cc:377] Logged in from keytab as kudu/127.25.92.194@KRBTEST.COM (short username kudu)
I20260805 23:13:24.273132 9610 webserver.cc:460] Webserver started at http://127.25.92.194:44771/ using document root <none> and password file <none>
I20260805 23:13:24.273849 9610 fs_manager.cc:359] Metadata directory not provided
I20260805 23:13:24.273953 9610 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:13:24.274228 9610 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:13:24.276481 9610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-1/data/instance:
uuid: "69b377dded7945329549bf5a64bf87b1"
format_stamp: "Formatted at 2026-08-05 23:13:24 on dist-test-slave-ss4n"
server_key: "a6fb1dbdbb49b2b0b0e8f2fdd7d0fd6d"
server_key_iv: "38dbd67ee19f383f3a91fe3b8aafd53e"
server_key_version: "encryptionkey@0"
I20260805 23:13:24.277138 9610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-1/wal/instance:
uuid: "69b377dded7945329549bf5a64bf87b1"
format_stamp: "Formatted at 2026-08-05 23:13:24 on dist-test-slave-ss4n"
server_key: "a6fb1dbdbb49b2b0b0e8f2fdd7d0fd6d"
server_key_iv: "38dbd67ee19f383f3a91fe3b8aafd53e"
server_key_version: "encryptionkey@0"
I20260805 23:13:24.281435 9610 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260805 23:13:24.284227 9625 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:13:24.285676 9610 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:13:24.285810 9610 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-1/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-1/wal
uuid: "69b377dded7945329549bf5a64bf87b1"
format_stamp: "Formatted at 2026-08-05 23:13:24 on dist-test-slave-ss4n"
server_key: "a6fb1dbdbb49b2b0b0e8f2fdd7d0fd6d"
server_key_iv: "38dbd67ee19f383f3a91fe3b8aafd53e"
server_key_version: "encryptionkey@0"
I20260805 23:13:24.285959 9610 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:13:24.304955 9610 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:13:24.308792 9610 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:13:24.309021 9610 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:13:24.309733 9610 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:13:24.310660 9610 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:13:24.310737 9610 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:13:24.310814 9610 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:13:24.310866 9610 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:13:24.321396 9610 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.194:44511
I20260805 23:13:24.321442 9740 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.194:44511 every 8 connection(s)
I20260805 23:13:24.322525 9610 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-1/data/info.pb
I20260805 23:13:24.327702 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 9610
I20260805 23:13:24.327879 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-1/wal/instance
I20260805 23:13:24.328212 25971 external_mini_cluster.cc:1428] Setting key 8cd137979163989a9ac2d8d7fdfad747
Aug 05 23:13:24 dist-test-slave-ss4n krb5kdc[9361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971604, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.194@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.25.92.195@KRBTEST.COM; defaulting to no policy
I20260805 23:13:24.340351 9449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:24.324601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.194:50055 (local address 127.25.92.254:34379)
0805 23:13:24.324801 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0805 23:13:24.324806 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:13:24.325809 (+ 1003us) server_negotiation.cc:407] Connection header received
0805 23:13:24.327277 (+ 1468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:13:24.327280 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:13:24.327397 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:13:24.327518 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:13:24.329058 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:13:24.329798 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:13:24.331172 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:13:24.331676 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:13:24.331820 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:13:24.335459 (+ 3639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:13:24.335467 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:13:24.335474 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:13:24.335519 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:13:24.337562 (+ 2043us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:13:24.338195 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:13:24.338202 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:13:24.338207 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:13:24.338319 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:13:24.338805 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:13:24.338811 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:13:24.338813 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:13:24.339154 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:13:24.339282 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0805 23:13:24.339926 (+ 644us) server_negotiation.cc:299] Negotiation successful
0805 23:13:24.340107 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260805 23:13:24.341142 9743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:24.325061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:34379 (local address 127.25.92.194:50055)
0805 23:13:24.325635 (+ 574us) negotiation.cc:107] Waiting for socket to connect
0805 23:13:24.325690 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0805 23:13:24.326993 (+ 1303us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:13:24.327689 (+ 696us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:13:24.327697 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:13:24.328223 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:13:24.328904 (+ 681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:13:24.328923 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:13:24.329999 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:13:24.330002 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:13:24.331012 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:13:24.331024 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:13:24.332182 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:13:24.332188 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:13:24.332475 (+ 287us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:13:24.333468 (+ 993us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:13:24.333611 (+ 143us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:13:24.335222 (+ 1611us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:13:24.337737 (+ 2515us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:13:24.337745 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:13:24.337765 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:13:24.338058 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:13:24.338464 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:13:24.338467 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:13:24.338468 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:13:24.338684 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:13:24.339284 (+ 600us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:13:24.339290 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:13:24.339638 (+ 348us) client_negotiation.cc:769] Sending connection context
0805 23:13:24.339817 (+ 179us) client_negotiation.cc:240] Negotiation successful
0805 23:13:24.340192 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":121,"threads_started":1}
I20260805 23:13:24.342549 9741 heartbeater.cc:347] Connected to a master server at 127.25.92.254:34379
I20260805 23:13:24.342989 9741 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.92.195@KRBTEST.COM" created.
I20260805 23:13:24.343653 9741 heartbeater.cc:511] Master 127.25.92.254:34379 requested a full tablet report, sending...
I20260805 23:13:24.345039 9408 ts_manager.cc:194] Registered new tserver with Master: 69b377dded7945329549bf5a64bf87b1 (127.25.92.194:44511)
I20260805 23:13:24.345817 9408 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.92.194@KRBTEST.COM'} at 127.25.92.194:50055
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.92.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.195.keytab.
Entry for principal kudu/127.25.92.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.195.keytab.
I20260805 23:13:24.362047 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.92.195:0
--local_ip_for_outbound_sockets=127.25.92.195
--webserver_interface=127.25.92.195
--webserver_port=0
--tserver_master_addrs=127.25.92.254:34379
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.195.keytab
--principal=kudu/127.25.92.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:33407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:13:24.476377 9746 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:13:24.476663 9746 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:13:24.476709 9746 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:13:24.480914 9746 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:13:24.480983 9746 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:13:24.481050 9746 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.92.195
I20260805 23:13:24.486400 9746 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:33407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.195.keytab
--principal=kudu/127.25.92.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.92.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.92.195
--webserver_port=0
--tserver_master_addrs=127.25.92.254:34379
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9746
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.92.195
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:13:24.487485 9746 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:13:24.488601 9746 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:13:24.498735 9755 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:13:24.498735 9751 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:13:24.498843 9746 server_base.cc:1034] running on GCE node
W20260805 23:13:24.499120 9753 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:13:24.499554 9746 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:13:24.500269 9746 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:13:24.501459 9746 hybrid_clock.cc:648] HybridClock initialized: now 1785971604501439 us; error 34 us; skew 500 ppm
Aug 05 23:13:24 dist-test-slave-ss4n krb5kdc[9361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971604, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:13:24.504684 9746 init.cc:377] Logged in from keytab as kudu/127.25.92.195@KRBTEST.COM (short username kudu)
I20260805 23:13:24.505733 9746 webserver.cc:460] Webserver started at http://127.25.92.195:39537/ using document root <none> and password file <none>
I20260805 23:13:24.506376 9746 fs_manager.cc:359] Metadata directory not provided
I20260805 23:13:24.506448 9746 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:13:24.506636 9746 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:13:24.509047 9746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-2/data/instance:
uuid: "fd89afa0150a4437b4f2ea95593ccaf0"
format_stamp: "Formatted at 2026-08-05 23:13:24 on dist-test-slave-ss4n"
server_key: "dad15bf071517ab654fa7c853f5edbfb"
server_key_iv: "3cf942aaac65f2be89eed7fbce6f0497"
server_key_version: "encryptionkey@0"
I20260805 23:13:24.509779 9746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-2/wal/instance:
uuid: "fd89afa0150a4437b4f2ea95593ccaf0"
format_stamp: "Formatted at 2026-08-05 23:13:24 on dist-test-slave-ss4n"
server_key: "dad15bf071517ab654fa7c853f5edbfb"
server_key_iv: "3cf942aaac65f2be89eed7fbce6f0497"
server_key_version: "encryptionkey@0"
I20260805 23:13:24.514295 9746 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260805 23:13:24.517050 9762 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:13:24.518738 9746 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260805 23:13:24.518854 9746 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-2/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-2/wal
uuid: "fd89afa0150a4437b4f2ea95593ccaf0"
format_stamp: "Formatted at 2026-08-05 23:13:24 on dist-test-slave-ss4n"
server_key: "dad15bf071517ab654fa7c853f5edbfb"
server_key_iv: "3cf942aaac65f2be89eed7fbce6f0497"
server_key_version: "encryptionkey@0"
I20260805 23:13:24.518962 9746 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:13:24.534006 9746 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:13:24.537675 9746 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:13:24.537906 9746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:13:24.538578 9746 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:13:24.539595 9746 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:13:24.539670 9746 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:13:24.539750 9746 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:13:24.539798 9746 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:13:24.550086 9746 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.195:41629
I20260805 23:13:24.550123 9880 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.195:41629 every 8 connection(s)
I20260805 23:13:24.551105 9746 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-2/data/info.pb
I20260805 23:13:24.560037 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 9746
I20260805 23:13:24.560231 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-2/wal/instance
I20260805 23:13:24.560562 25971 external_mini_cluster.cc:1428] Setting key f0fb71da5b7b509c7ed056af1574f1d1
Aug 05 23:13:24 dist-test-slave-ss4n krb5kdc[9361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971604, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.195@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM
I20260805 23:13:24.566350 9449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:24.553138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.195:46121 (local address 127.25.92.254:34379)
0805 23:13:24.553308 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0805 23:13:24.553314 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:13:24.554349 (+ 1035us) server_negotiation.cc:407] Connection header received
0805 23:13:24.555693 (+ 1344us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:13:24.555696 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:13:24.555749 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:13:24.555836 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:13:24.557019 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:13:24.557688 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:13:24.558904 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:13:24.559130 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:13:24.559227 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:13:24.562318 (+ 3091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:13:24.562322 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:13:24.562325 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:13:24.562362 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:13:24.563999 (+ 1637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:13:24.564600 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:13:24.564604 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:13:24.564606 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:13:24.564718 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:13:24.565144 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:13:24.565147 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:13:24.565149 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:13:24.565361 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:13:24.565472 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0805 23:13:24.566044 (+ 572us) server_negotiation.cc:299] Negotiation successful
0805 23:13:24.566186 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260805 23:13:24.567207 9883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:24.553704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:34379 (local address 127.25.92.195:46121)
0805 23:13:24.554188 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0805 23:13:24.554232 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0805 23:13:24.555452 (+ 1220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:13:24.556000 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:13:24.556006 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:13:24.556385 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:13:24.556854 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:13:24.556866 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:13:24.557842 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:13:24.557845 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:13:24.558763 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:13:24.558773 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:13:24.559288 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:13:24.559291 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:13:24.559436 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:13:24.560167 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:13:24.560183 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:13:24.562126 (+ 1943us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:13:24.564152 (+ 2026us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:13:24.564159 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:13:24.564177 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:13:24.564483 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:13:24.564821 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:13:24.564824 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:13:24.564826 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:13:24.564978 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:13:24.565472 (+ 494us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:13:24.565519 (+ 47us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:13:24.565801 (+ 282us) client_negotiation.cc:769] Sending connection context
0805 23:13:24.566004 (+ 203us) client_negotiation.cc:240] Negotiation successful
0805 23:13:24.566355 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":123,"threads_started":1}
I20260805 23:13:24.568359 9881 heartbeater.cc:347] Connected to a master server at 127.25.92.254:34379
I20260805 23:13:24.568629 9881 heartbeater.cc:464] Registering TS with master...
I20260805 23:13:24.569165 9881 heartbeater.cc:511] Master 127.25.92.254:34379 requested a full tablet report, sending...
I20260805 23:13:24.570359 9408 ts_manager.cc:194] Registered new tserver with Master: fd89afa0150a4437b4f2ea95593ccaf0 (127.25.92.195:41629)
I20260805 23:13:24.570959 9408 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.92.195@KRBTEST.COM'} at 127.25.92.195:46121
I20260805 23:13:24.575234 25971 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260805 23:13:24.586426 9449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:24.577363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44554 (local address 127.25.92.254:34379)
0805 23:13:24.577531 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0805 23:13:24.577537 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:13:24.577748 (+ 211us) server_negotiation.cc:407] Connection header received
0805 23:13:24.577908 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:13:24.577911 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:13:24.577966 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:13:24.578061 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:13:24.578824 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:13:24.579362 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:13:24.580274 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:13:24.580514 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:13:24.580838 (+ 324us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:13:24.582874 (+ 2036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:13:24.582878 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:13:24.582881 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:13:24.582918 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:13:24.584394 (+ 1476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:13:24.584901 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:13:24.584906 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:13:24.584908 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:13:24.585014 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:13:24.585325 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:13:24.585330 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:13:24.585332 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:13:24.585671 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:13:24.585827 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0805 23:13:24.586109 (+ 282us) server_negotiation.cc:299] Negotiation successful
0805 23:13:24.586242 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260805 23:13:24.590076 9408 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44554:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260805 23:13:24.593375 9408 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260805 23:13:24.609920 9894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:24.603302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.193:45305 (local address 127.0.0.1:52670)
0805 23:13:24.604721 (+ 1419us) negotiation.cc:107] Waiting for socket to connect
0805 23:13:24.604734 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0805 23:13:24.604856 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:13:24.605548 (+ 692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:13:24.605551 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:13:24.605570 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:13:24.605659 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:13:24.605663 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:13:24.606732 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:13:24.606738 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:13:24.608210 (+ 1472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:13:24.608222 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:13:24.609306 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:13:24.609309 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:13:24.609432 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:13:24.609470 (+ 38us) client_negotiation.cc:769] Sending connection context
0805 23:13:24.609612 (+ 142us) client_negotiation.cc:240] Negotiation successful
0805 23:13:24.609732 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1332,"mutex_wait_us":177,"thread_start_us":104,"threads_started":1}
I20260805 23:13:24.610291 9895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:24.603175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39998 (local address 127.25.92.194:44511)
0805 23:13:24.604449 (+ 1274us) server_negotiation.cc:206] Beginning negotiation
0805 23:13:24.604461 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0805 23:13:24.604485 (+ 24us) server_negotiation.cc:407] Connection header received
0805 23:13:24.604567 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:13:24.604575 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:13:24.604785 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:13:24.604943 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:13:24.605259 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:13:24.606553 (+ 1294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:13:24.608688 (+ 2135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:13:24.609290 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:13:24.609397 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:13:24.609571 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0805 23:13:24.609841 (+ 270us) server_negotiation.cc:299] Negotiation successful
0805 23:13:24.609948 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1164,"thread_start_us":187,"threads_started":1}
I20260805 23:13:24.610392 9897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:24.603885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52670 (local address 127.25.92.193:45305)
0805 23:13:24.604552 (+ 667us) server_negotiation.cc:206] Beginning negotiation
0805 23:13:24.604564 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0805 23:13:24.604835 (+ 271us) server_negotiation.cc:407] Connection header received
0805 23:13:24.605007 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:13:24.605013 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:13:24.605213 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:13:24.605353 (+ 140us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:13:24.605781 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:13:24.606580 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:13:24.608348 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:13:24.609164 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:13:24.609284 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:13:24.609445 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0805 23:13:24.610138 (+ 693us) server_negotiation.cc:299] Negotiation successful
0805 23:13:24.610233 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":559,"thread_start_us":92,"threads_started":1}
I20260805 23:13:24.610388 9893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:24.603022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.194:44511 (local address 127.0.0.1:39998)
0805 23:13:24.603815 (+ 793us) negotiation.cc:107] Waiting for socket to connect
0805 23:13:24.603870 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0805 23:13:24.604013 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:13:24.604948 (+ 935us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:13:24.604954 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:13:24.604978 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:13:24.605122 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:13:24.605129 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:13:24.606726 (+ 1597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:13:24.606730 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:13:24.608520 (+ 1790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:13:24.608532 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:13:24.609588 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:13:24.609593 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:13:24.609684 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:13:24.609707 (+ 23us) client_negotiation.cc:769] Sending connection context
0805 23:13:24.609786 (+ 79us) client_negotiation.cc:240] Negotiation successful
0805 23:13:24.610233 (+ 447us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":605,"mutex_wait_us":217,"thread_start_us":101,"threads_started":1}
I20260805 23:13:24.611783 9898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:24.603881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42064 (local address 127.25.92.195:41629)
0805 23:13:24.604348 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0805 23:13:24.604362 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0805 23:13:24.604557 (+ 195us) server_negotiation.cc:407] Connection header received
0805 23:13:24.604651 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:13:24.604656 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:13:24.604800 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:13:24.604943 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:13:24.605432 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:13:24.606592 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:13:24.608662 (+ 2070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:13:24.609457 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:13:24.609645 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:13:24.609784 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0805 23:13:24.611523 (+ 1739us) server_negotiation.cc:299] Negotiation successful
0805 23:13:24.611654 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":128,"threads_started":1}
I20260805 23:13:24.610790 9896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:24.603241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.195:41629 (local address 127.0.0.1:42064)
0805 23:13:24.604328 (+ 1087us) negotiation.cc:107] Waiting for socket to connect
0805 23:13:24.604342 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0805 23:13:24.604446 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:13:24.605138 (+ 692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:13:24.605141 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:13:24.605158 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:13:24.605290 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:13:24.605298 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:13:24.606726 (+ 1428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:13:24.606730 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:13:24.608520 (+ 1790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:13:24.608532 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:13:24.609847 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:13:24.609850 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:13:24.609937 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:13:24.609963 (+ 26us) client_negotiation.cc:769] Sending connection context
0805 23:13:24.610580 (+ 617us) client_negotiation.cc:240] Negotiation successful
0805 23:13:24.610661 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1018,"mutex_wait_us":286,"thread_start_us":110,"threads_started":1}
I20260805 23:13:24.613911 9536 tablet_service.cc:1467] Processing CreateTablet for tablet 084470555dc148c48d79b194e2e60621 (DEFAULT_TABLE table=test-table [id=0c49d00738f14a6eaee90131887926d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:13:24.614379 9810 tablet_service.cc:1467] Processing CreateTablet for tablet 084470555dc148c48d79b194e2e60621 (DEFAULT_TABLE table=test-table [id=0c49d00738f14a6eaee90131887926d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:13:24.615036 9536 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 084470555dc148c48d79b194e2e60621. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:13:24.615414 9810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 084470555dc148c48d79b194e2e60621. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:13:24.621239 9899 tablet_bootstrap.cc:492] T 084470555dc148c48d79b194e2e60621 P 85d51d37bc304498873dd204279742da: Bootstrap starting.
I20260805 23:13:24.621301 9900 tablet_bootstrap.cc:492] T 084470555dc148c48d79b194e2e60621 P fd89afa0150a4437b4f2ea95593ccaf0: Bootstrap starting.
I20260805 23:13:24.623847 9899 tablet_bootstrap.cc:654] T 084470555dc148c48d79b194e2e60621 P 85d51d37bc304498873dd204279742da: Neither blocks nor log segments found. Creating new log.
I20260805 23:13:24.623867 9900 tablet_bootstrap.cc:654] T 084470555dc148c48d79b194e2e60621 P fd89afa0150a4437b4f2ea95593ccaf0: Neither blocks nor log segments found. Creating new log.
I20260805 23:13:24.623978 9674 tablet_service.cc:1467] Processing CreateTablet for tablet 084470555dc148c48d79b194e2e60621 (DEFAULT_TABLE table=test-table [id=0c49d00738f14a6eaee90131887926d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:13:24.624861 9899 log.cc:824] T 084470555dc148c48d79b194e2e60621 P 85d51d37bc304498873dd204279742da: Log is configured to *not* fsync() on all Append() calls
I20260805 23:13:24.625161 9900 log.cc:824] T 084470555dc148c48d79b194e2e60621 P fd89afa0150a4437b4f2ea95593ccaf0: Log is configured to *not* fsync() on all Append() calls
I20260805 23:13:24.625248 9674 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 084470555dc148c48d79b194e2e60621. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:13:24.627413 9899 tablet_bootstrap.cc:492] T 084470555dc148c48d79b194e2e60621 P 85d51d37bc304498873dd204279742da: No bootstrap required, opened a new log
I20260805 23:13:24.627413 9900 tablet_bootstrap.cc:492] T 084470555dc148c48d79b194e2e60621 P fd89afa0150a4437b4f2ea95593ccaf0: No bootstrap required, opened a new log
I20260805 23:13:24.627643 9899 ts_tablet_manager.cc:1397] T 084470555dc148c48d79b194e2e60621 P 85d51d37bc304498873dd204279742da: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260805 23:13:24.627636 9900 ts_tablet_manager.cc:1397] T 084470555dc148c48d79b194e2e60621 P fd89afa0150a4437b4f2ea95593ccaf0: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260805 23:13:24.631785 9904 tablet_bootstrap.cc:492] T 084470555dc148c48d79b194e2e60621 P 69b377dded7945329549bf5a64bf87b1: Bootstrap starting.
I20260805 23:13:24.631783 9900 raft_consensus.cc:348] T 084470555dc148c48d79b194e2e60621 P fd89afa0150a4437b4f2ea95593ccaf0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69b377dded7945329549bf5a64bf87b1" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 44511 } } peers { permanent_uuid: "fd89afa0150a4437b4f2ea95593ccaf0" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 41629 } } peers { permanent_uuid: "85d51d37bc304498873dd204279742da" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 45305 } }
I20260805 23:13:24.632227 9900 raft_consensus.cc:374] T 084470555dc148c48d79b194e2e60621 P fd89afa0150a4437b4f2ea95593ccaf0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:13:24.632344 9900 raft_consensus.cc:729] T 084470555dc148c48d79b194e2e60621 P fd89afa0150a4437b4f2ea95593ccaf0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd89afa0150a4437b4f2ea95593ccaf0, State: Initialized, Role: FOLLOWER
I20260805 23:13:24.632282 9899 raft_consensus.cc:348] T 084470555dc148c48d79b194e2e60621 P 85d51d37bc304498873dd204279742da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69b377dded7945329549bf5a64bf87b1" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 44511 } } peers { permanent_uuid: "fd89afa0150a4437b4f2ea95593ccaf0" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 41629 } } peers { permanent_uuid: "85d51d37bc304498873dd204279742da" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 45305 } }
I20260805 23:13:24.632797 9899 raft_consensus.cc:374] T 084470555dc148c48d79b194e2e60621 P 85d51d37bc304498873dd204279742da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:13:24.632863 9899 raft_consensus.cc:729] T 084470555dc148c48d79b194e2e60621 P 85d51d37bc304498873dd204279742da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85d51d37bc304498873dd204279742da, State: Initialized, Role: FOLLOWER
I20260805 23:13:24.632939 9900 consensus_queue.cc:260] T 084470555dc148c48d79b194e2e60621 P fd89afa0150a4437b4f2ea95593ccaf0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69b377dded7945329549bf5a64bf87b1" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 44511 } } peers { permanent_uuid: "fd89afa0150a4437b4f2ea95593ccaf0" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 41629 } } peers { permanent_uuid: "85d51d37bc304498873dd204279742da" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 45305 } }
I20260805 23:13:24.633322 9899 consensus_queue.cc:260] T 084470555dc148c48d79b194e2e60621 P 85d51d37bc304498873dd204279742da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69b377dded7945329549bf5a64bf87b1" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 44511 } } peers { permanent_uuid: "fd89afa0150a4437b4f2ea95593ccaf0" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 41629 } } peers { permanent_uuid: "85d51d37bc304498873dd204279742da" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 45305 } }
I20260805 23:13:24.634158 9900 ts_tablet_manager.cc:1428] T 084470555dc148c48d79b194e2e60621 P fd89afa0150a4437b4f2ea95593ccaf0: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260805 23:13:24.634253 9899 ts_tablet_manager.cc:1428] T 084470555dc148c48d79b194e2e60621 P 85d51d37bc304498873dd204279742da: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260805 23:13:24.634559 9605 heartbeater.cc:503] Master 127.25.92.254:34379 was elected leader, sending a full tablet report...
I20260805 23:13:24.634578 9881 heartbeater.cc:503] Master 127.25.92.254:34379 was elected leader, sending a full tablet report...
I20260805 23:13:24.634955 9904 tablet_bootstrap.cc:654] T 084470555dc148c48d79b194e2e60621 P 69b377dded7945329549bf5a64bf87b1: Neither blocks nor log segments found. Creating new log.
I20260805 23:13:24.635896 9904 log.cc:824] T 084470555dc148c48d79b194e2e60621 P 69b377dded7945329549bf5a64bf87b1: Log is configured to *not* fsync() on all Append() calls
I20260805 23:13:24.638417 9904 tablet_bootstrap.cc:492] T 084470555dc148c48d79b194e2e60621 P 69b377dded7945329549bf5a64bf87b1: No bootstrap required, opened a new log
I20260805 23:13:24.638701 9904 ts_tablet_manager.cc:1397] T 084470555dc148c48d79b194e2e60621 P 69b377dded7945329549bf5a64bf87b1: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260805 23:13:24.642562 9904 raft_consensus.cc:348] T 084470555dc148c48d79b194e2e60621 P 69b377dded7945329549bf5a64bf87b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69b377dded7945329549bf5a64bf87b1" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 44511 } } peers { permanent_uuid: "fd89afa0150a4437b4f2ea95593ccaf0" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 41629 } } peers { permanent_uuid: "85d51d37bc304498873dd204279742da" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 45305 } }
I20260805 23:13:24.642833 9904 raft_consensus.cc:374] T 084470555dc148c48d79b194e2e60621 P 69b377dded7945329549bf5a64bf87b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:13:24.642869 9904 raft_consensus.cc:729] T 084470555dc148c48d79b194e2e60621 P 69b377dded7945329549bf5a64bf87b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69b377dded7945329549bf5a64bf87b1, State: Initialized, Role: FOLLOWER
I20260805 23:13:24.643298 9904 consensus_queue.cc:260] T 084470555dc148c48d79b194e2e60621 P 69b377dded7945329549bf5a64bf87b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69b377dded7945329549bf5a64bf87b1" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 44511 } } peers { permanent_uuid: "fd89afa0150a4437b4f2ea95593ccaf0" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 41629 } } peers { permanent_uuid: "85d51d37bc304498873dd204279742da" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 45305 } }
I20260805 23:13:24.643965 9741 heartbeater.cc:503] Master 127.25.92.254:34379 was elected leader, sending a full tablet report...
I20260805 23:13:24.644313 9904 ts_tablet_manager.cc:1428] T 084470555dc148c48d79b194e2e60621 P 69b377dded7945329549bf5a64bf87b1: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260805 23:13:24.670311 9905 raft_consensus.cc:482] T 084470555dc148c48d79b194e2e60621 P fd89afa0150a4437b4f2ea95593ccaf0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 23:13:24.670614 9905 raft_consensus.cc:504] T 084470555dc148c48d79b194e2e60621 P fd89afa0150a4437b4f2ea95593ccaf0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69b377dded7945329549bf5a64bf87b1" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 44511 } } peers { permanent_uuid: "fd89afa0150a4437b4f2ea95593ccaf0" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 41629 } } peers { permanent_uuid: "85d51d37bc304498873dd204279742da" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 45305 } }
I20260805 23:13:24.671897 9905 leader_election.cc:288] T 084470555dc148c48d79b194e2e60621 P fd89afa0150a4437b4f2ea95593ccaf0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 69b377dded7945329549bf5a64bf87b1 (127.25.92.194:44511), 85d51d37bc304498873dd204279742da (127.25.92.193:45305)
I20260805 23:13:24.676201 9883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:24.672171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.193:45305 (local address 127.25.92.195:58941)
0805 23:13:24.672318 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0805 23:13:24.672335 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0805 23:13:24.672444 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:13:24.672781 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:13:24.672785 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:13:24.672811 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:13:24.672946 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:13:24.672953 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:13:24.674080 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:13:24.674083 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:13:24.674937 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:13:24.674946 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:13:24.675746 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:13:24.675749 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:13:24.675864 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:13:24.675892 (+ 28us) client_negotiation.cc:769] Sending connection context
0805 23:13:24.675984 (+ 92us) client_negotiation.cc:240] Negotiation successful
0805 23:13:24.676060 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260805 23:13:24.676201 9897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:24.672355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.195:58941 (local address 127.25.92.193:45305)
0805 23:13:24.672498 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0805 23:13:24.672506 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0805 23:13:24.672523 (+ 17us) server_negotiation.cc:407] Connection header received
0805 23:13:24.672589 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:13:24.672594 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:13:24.672651 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:13:24.672748 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:13:24.673107 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:13:24.673971 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:13:24.675080 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:13:24.675618 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:13:24.675754 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:13:24.675811 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0805 23:13:24.676004 (+ 193us) server_negotiation.cc:299] Negotiation successful
0805 23:13:24.676090 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260805 23:13:24.676787 9895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:24.672286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.195:46485 (local address 127.25.92.194:44511)
0805 23:13:24.672425 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0805 23:13:24.672430 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:13:24.673194 (+ 764us) server_negotiation.cc:407] Connection header received
0805 23:13:24.673402 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:13:24.673406 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:13:24.673459 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:13:24.673586 (+ 127us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:13:24.673843 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:13:24.674471 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:13:24.675602 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:13:24.676160 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:13:24.676418 (+ 258us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:13:24.676479 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0805 23:13:24.676587 (+ 108us) server_negotiation.cc:299] Negotiation successful
0805 23:13:24.676663 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260805 23:13:24.676849 9909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:24.672174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.194:44511 (local address 127.25.92.195:46485)
0805 23:13:24.673071 (+ 897us) negotiation.cc:107] Waiting for socket to connect
0805 23:13:24.673088 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0805 23:13:24.673232 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:13:24.673600 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:13:24.673603 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:13:24.673621 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:13:24.673716 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:13:24.673720 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:13:24.674597 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:13:24.674602 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:13:24.675458 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:13:24.675467 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:13:24.676329 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:13:24.676334 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:13:24.676443 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:13:24.676467 (+ 24us) client_negotiation.cc:769] Sending connection context
0805 23:13:24.676561 (+ 94us) client_negotiation.cc:240] Negotiation successful
0805 23:13:24.676637 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":801,"mutex_wait_us":353,"thread_start_us":210,"threads_started":1}
I20260805 23:13:24.676934 9557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "084470555dc148c48d79b194e2e60621" candidate_uuid: "fd89afa0150a4437b4f2ea95593ccaf0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85d51d37bc304498873dd204279742da" is_pre_election: true
I20260805 23:13:24.677325 9557 raft_consensus.cc:2454] T 084470555dc148c48d79b194e2e60621 P 85d51d37bc304498873dd204279742da [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fd89afa0150a4437b4f2ea95593ccaf0 in term 0.
I20260805 23:13:24.677592 9694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "084470555dc148c48d79b194e2e60621" candidate_uuid: "fd89afa0150a4437b4f2ea95593ccaf0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69b377dded7945329549bf5a64bf87b1" is_pre_election: true
I20260805 23:13:24.677940 9694 raft_consensus.cc:2454] T 084470555dc148c48d79b194e2e60621 P 69b377dded7945329549bf5a64bf87b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fd89afa0150a4437b4f2ea95593ccaf0 in term 0.
I20260805 23:13:24.677933 9763 leader_election.cc:302] T 084470555dc148c48d79b194e2e60621 P fd89afa0150a4437b4f2ea95593ccaf0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85d51d37bc304498873dd204279742da, fd89afa0150a4437b4f2ea95593ccaf0; no voters:
I20260805 23:13:24.678246 9905 raft_consensus.cc:2781] T 084470555dc148c48d79b194e2e60621 P fd89afa0150a4437b4f2ea95593ccaf0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260805 23:13:24.678339 9905 raft_consensus.cc:482] T 084470555dc148c48d79b194e2e60621 P fd89afa0150a4437b4f2ea95593ccaf0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260805 23:13:24.678403 9905 raft_consensus.cc:3037] T 084470555dc148c48d79b194e2e60621 P fd89afa0150a4437b4f2ea95593ccaf0 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:13:24.679466 9905 raft_consensus.cc:504] T 084470555dc148c48d79b194e2e60621 P fd89afa0150a4437b4f2ea95593ccaf0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69b377dded7945329549bf5a64bf87b1" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 44511 } } peers { permanent_uuid: "fd89afa0150a4437b4f2ea95593ccaf0" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 41629 } } peers { permanent_uuid: "85d51d37bc304498873dd204279742da" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 45305 } }
I20260805 23:13:24.679873 9905 leader_election.cc:288] T 084470555dc148c48d79b194e2e60621 P fd89afa0150a4437b4f2ea95593ccaf0 [CANDIDATE]: Term 1 election: Requested vote from peers 69b377dded7945329549bf5a64bf87b1 (127.25.92.194:44511), 85d51d37bc304498873dd204279742da (127.25.92.193:45305)
I20260805 23:13:24.680220 9694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "084470555dc148c48d79b194e2e60621" candidate_uuid: "fd89afa0150a4437b4f2ea95593ccaf0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69b377dded7945329549bf5a64bf87b1"
I20260805 23:13:24.680382 9694 raft_consensus.cc:3037] T 084470555dc148c48d79b194e2e60621 P 69b377dded7945329549bf5a64bf87b1 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:13:24.680409 9557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "084470555dc148c48d79b194e2e60621" candidate_uuid: "fd89afa0150a4437b4f2ea95593ccaf0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85d51d37bc304498873dd204279742da"
I20260805 23:13:24.680548 9557 raft_consensus.cc:3037] T 084470555dc148c48d79b194e2e60621 P 85d51d37bc304498873dd204279742da [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:13:24.681954 9694 raft_consensus.cc:2454] T 084470555dc148c48d79b194e2e60621 P 69b377dded7945329549bf5a64bf87b1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd89afa0150a4437b4f2ea95593ccaf0 in term 1.
I20260805 23:13:24.682120 9557 raft_consensus.cc:2454] T 084470555dc148c48d79b194e2e60621 P 85d51d37bc304498873dd204279742da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd89afa0150a4437b4f2ea95593ccaf0 in term 1.
I20260805 23:13:24.682363 9766 leader_election.cc:302] T 084470555dc148c48d79b194e2e60621 P fd89afa0150a4437b4f2ea95593ccaf0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 69b377dded7945329549bf5a64bf87b1, fd89afa0150a4437b4f2ea95593ccaf0; no voters:
I20260805 23:13:24.682622 9905 raft_consensus.cc:2781] T 084470555dc148c48d79b194e2e60621 P fd89afa0150a4437b4f2ea95593ccaf0 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:13:24.682878 9905 raft_consensus.cc:686] T 084470555dc148c48d79b194e2e60621 P fd89afa0150a4437b4f2ea95593ccaf0 [term 1 LEADER]: Becoming Leader. State: Replica: fd89afa0150a4437b4f2ea95593ccaf0, State: Running, Role: LEADER
I20260805 23:13:24.683245 9905 consensus_queue.cc:237] T 084470555dc148c48d79b194e2e60621 P fd89afa0150a4437b4f2ea95593ccaf0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69b377dded7945329549bf5a64bf87b1" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 44511 } } peers { permanent_uuid: "fd89afa0150a4437b4f2ea95593ccaf0" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 41629 } } peers { permanent_uuid: "85d51d37bc304498873dd204279742da" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 45305 } }
I20260805 23:13:24.686702 9408 catalog_manager.cc:5495] T 084470555dc148c48d79b194e2e60621 P fd89afa0150a4437b4f2ea95593ccaf0 reported cstate change: term changed from 0 to 1, leader changed from <none> to fd89afa0150a4437b4f2ea95593ccaf0 (127.25.92.195). New cstate: current_term: 1 leader_uuid: "fd89afa0150a4437b4f2ea95593ccaf0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69b377dded7945329549bf5a64bf87b1" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 44511 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fd89afa0150a4437b4f2ea95593ccaf0" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 41629 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85d51d37bc304498873dd204279742da" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 45305 } health_report { overall_health: UNKNOWN } } }
I20260805 23:13:24.718166 9898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:24.714359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42068 (local address 127.25.92.195:41629)
0805 23:13:24.714488 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0805 23:13:24.714491 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 23:13:24.714513 (+ 22us) server_negotiation.cc:407] Connection header received
0805 23:13:24.714654 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:13:24.714656 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:13:24.714709 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:13:24.714798 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 23:13:24.715041 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:13:24.715597 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:13:24.716594 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:13:24.716813 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:13:24.716904 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:13:24.717233 (+ 329us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 23:13:24.717718 (+ 485us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 23:13:24.717813 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0805 23:13:24.717894 (+ 81us) server_negotiation.cc:299] Negotiation successful
0805 23:13:24.717953 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260805 23:13:24.724342 9694 raft_consensus.cc:1261] T 084470555dc148c48d79b194e2e60621 P 69b377dded7945329549bf5a64bf87b1 [term 1 FOLLOWER]: Refusing update from remote peer fd89afa0150a4437b4f2ea95593ccaf0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:13:24.724342 9557 raft_consensus.cc:1261] T 084470555dc148c48d79b194e2e60621 P 85d51d37bc304498873dd204279742da [term 1 FOLLOWER]: Refusing update from remote peer fd89afa0150a4437b4f2ea95593ccaf0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:13:24.725198 9905 consensus_queue.cc:1035] T 084470555dc148c48d79b194e2e60621 P fd89afa0150a4437b4f2ea95593ccaf0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "69b377dded7945329549bf5a64bf87b1" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 44511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:13:24.725418 9910 consensus_queue.cc:1035] T 084470555dc148c48d79b194e2e60621 P fd89afa0150a4437b4f2ea95593ccaf0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85d51d37bc304498873dd204279742da" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 45305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:13:24.744746 9406 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44554:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:13:24.744963 9406 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44554:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:13:24.747757 9406 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 242b9070a63b4c049d7c7563d64f021b: Sending DeleteTablet for 3 replicas of tablet 084470555dc148c48d79b194e2e60621
I20260805 23:13:24.748627 9810 tablet_service.cc:1514] Processing DeleteTablet for tablet 084470555dc148c48d79b194e2e60621 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:13:24 UTC) from {username='kudu', principal='kudu/127.25.92.254@KRBTEST.COM'} at 127.0.0.1:42064
I20260805 23:13:24.748694 9536 tablet_service.cc:1514] Processing DeleteTablet for tablet 084470555dc148c48d79b194e2e60621 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:13:24 UTC) from {username='kudu', principal='kudu/127.25.92.254@KRBTEST.COM'} at 127.0.0.1:52670
I20260805 23:13:24.748690 9674 tablet_service.cc:1514] Processing DeleteTablet for tablet 084470555dc148c48d79b194e2e60621 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:13:24 UTC) from {username='kudu', principal='kudu/127.25.92.254@KRBTEST.COM'} at 127.0.0.1:39998
I20260805 23:13:24.749375 9927 tablet_replica.cc:330] T 084470555dc148c48d79b194e2e60621 P 69b377dded7945329549bf5a64bf87b1: stopping tablet replica
I20260805 23:13:24.749370 9925 tablet_replica.cc:330] T 084470555dc148c48d79b194e2e60621 P 85d51d37bc304498873dd204279742da: stopping tablet replica
I20260805 23:13:24.749657 9926 tablet_replica.cc:330] T 084470555dc148c48d79b194e2e60621 P fd89afa0150a4437b4f2ea95593ccaf0: stopping tablet replica
I20260805 23:13:24.749719 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 9472
I20260805 23:13:24.751052 9927 raft_consensus.cc:2229] T 084470555dc148c48d79b194e2e60621 P 69b377dded7945329549bf5a64bf87b1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 23:13:24.751410 9927 raft_consensus.cc:2258] T 084470555dc148c48d79b194e2e60621 P 69b377dded7945329549bf5a64bf87b1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:13:24.752420 9926 raft_consensus.cc:2229] T 084470555dc148c48d79b194e2e60621 P fd89afa0150a4437b4f2ea95593ccaf0 [term 1 LEADER]: Raft consensus shutting down.
I20260805 23:13:24.752848 9926 raft_consensus.cc:2258] T 084470555dc148c48d79b194e2e60621 P fd89afa0150a4437b4f2ea95593ccaf0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:13:24.753103 9927 ts_tablet_manager.cc:1905] T 084470555dc148c48d79b194e2e60621 P 69b377dded7945329549bf5a64bf87b1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:13:24.755975 9927 ts_tablet_manager.cc:1918] T 084470555dc148c48d79b194e2e60621 P 69b377dded7945329549bf5a64bf87b1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 23:13:24.756117 9927 log.cc:1196] T 084470555dc148c48d79b194e2e60621 P 69b377dded7945329549bf5a64bf87b1: Deleting WAL directory at /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-1/wal/wals/084470555dc148c48d79b194e2e60621
I20260805 23:13:24.756532 9927 ts_tablet_manager.cc:1939] T 084470555dc148c48d79b194e2e60621 P 69b377dded7945329549bf5a64bf87b1: Deleting consensus metadata
I20260805 23:13:24.757735 9926 ts_tablet_manager.cc:1905] T 084470555dc148c48d79b194e2e60621 P fd89afa0150a4437b4f2ea95593ccaf0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:13:24.758018 9393 catalog_manager.cc:4841] TS 69b377dded7945329549bf5a64bf87b1 (127.25.92.194:44511): tablet 084470555dc148c48d79b194e2e60621 (table test-table [id=0c49d00738f14a6eaee90131887926d0]) successfully deleted
I20260805 23:13:24.760066 9926 ts_tablet_manager.cc:1918] T 084470555dc148c48d79b194e2e60621 P fd89afa0150a4437b4f2ea95593ccaf0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 23:13:24.760195 9926 log.cc:1196] T 084470555dc148c48d79b194e2e60621 P fd89afa0150a4437b4f2ea95593ccaf0: Deleting WAL directory at /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-2/wal/wals/084470555dc148c48d79b194e2e60621
I20260805 23:13:24.760527 9926 ts_tablet_manager.cc:1939] T 084470555dc148c48d79b194e2e60621 P fd89afa0150a4437b4f2ea95593ccaf0: Deleting consensus metadata
I20260805 23:13:24.761850 9391 catalog_manager.cc:4841] TS fd89afa0150a4437b4f2ea95593ccaf0 (127.25.92.195:41629): tablet 084470555dc148c48d79b194e2e60621 (table test-table [id=0c49d00738f14a6eaee90131887926d0]) successfully deleted
W20260805 23:13:24.761884 9390 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.25.92.193:45305 (error 108)
I20260805 23:13:24.762440 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 9610
I20260805 23:13:24.764436 9896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:24.764145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.193:45305 (local address 127.0.0.1:52678)
0805 23:13:24.764280 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0805 23:13:24.764355 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.193:45305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260805 23:13:24.764649 9390 catalog_manager.cc:4584] TS 85d51d37bc304498873dd204279742da (127.25.92.193:45305): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 084470555dc148c48d79b194e2e60621: Network error: Client connection negotiation failed: client connection to 127.25.92.193:45305: connect: Connection refused (error 111)
I20260805 23:13:24.771420 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 9746
I20260805 23:13:24.777657 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 9374
I20260805 23:13:44.785789 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.254:34379
--webserver_interface=127.25.92.254
--webserver_port=40893
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab
--principal=kudu/127.25.92.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:33407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.92.254:34379
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:13:44.897424 9929 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:13:44.897714 9929 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:13:44.897754 9929 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:13:44.901878 9929 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260805 23:13:44.901942 9929 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260805 23:13:44.901958 9929 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260805 23:13:44.901973 9929 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:13:44.901990 9929 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:13:44.902069 9929 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:13:44.902100 9929 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:13:44.902141 9929 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:13:44.907557 9929 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:33407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.25.92.254:34379
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab
--principal=kudu/127.25.92.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.254:34379
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.92.254
--webserver_port=40893
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9929
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:13:44.908793 9929 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:13:44.909912 9929 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:13:44.923962 9938 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:13:44.923969 9935 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:13:44.924033 9936 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:13:44.924362 9929 server_base.cc:1034] running on GCE node
I20260805 23:13:44.925001 9929 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:13:44.926276 9929 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:13:44.927469 9929 hybrid_clock.cc:648] HybridClock initialized: now 1785971624927460 us; error 30 us; skew 500 ppm
Aug 05 23:13:44 dist-test-slave-ss4n krb5kdc[9361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971624, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:13:44.935451 9929 init.cc:377] Logged in from keytab as kudu/127.25.92.254@KRBTEST.COM (short username kudu)
I20260805 23:13:44.936515 9929 webserver.cc:460] Webserver started at http://127.25.92.254:40893/ using document root <none> and password file <none>
I20260805 23:13:44.937145 9929 fs_manager.cc:359] Metadata directory not provided
I20260805 23:13:44.937263 9929 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:13:44.941349 9929 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:13:44.943602 9946 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:13:44.944809 9929 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260805 23:13:44.944963 9929 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/master-0/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/master-0/wal
uuid: "242b9070a63b4c049d7c7563d64f021b"
format_stamp: "Formatted at 2026-08-05 23:13:23 on dist-test-slave-ss4n"
server_key: "9cde2619d4310f38000f1ef33c06a78a"
server_key_iv: "f6299010fe621bb41c17e89177c7cdaa"
server_key_version: "encryptionkey@0"
I20260805 23:13:44.945650 9929 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:13:44.965802 9929 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:13:44.969318 9929 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:13:44.969563 9929 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:13:44.977391 9929 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.254:34379
I20260805 23:13:44.977751 9998 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.254:34379 every 8 connection(s)
I20260805 23:13:44.978734 9929 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
I20260805 23:13:44.983275 9999 sys_catalog.cc:263] Verifying existing consensus state
I20260805 23:13:44.983345 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 9929
I20260805 23:13:44.983778 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.92.193:45305
--local_ip_for_outbound_sockets=127.25.92.193
--tserver_master_addrs=127.25.92.254:34379
--webserver_port=43515
--webserver_interface=127.25.92.193
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.193.keytab
--principal=kudu/127.25.92.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:33407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260805 23:13:44.985143 9999 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 242b9070a63b4c049d7c7563d64f021b: Bootstrap starting.
I20260805 23:13:45.003595 9999 log.cc:824] T 00000000000000000000000000000000 P 242b9070a63b4c049d7c7563d64f021b: Log is configured to *not* fsync() on all Append() calls
I20260805 23:13:45.012519 9999 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 242b9070a63b4c049d7c7563d64f021b: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260805 23:13:45.012948 9999 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 242b9070a63b4c049d7c7563d64f021b: Bootstrap complete.
I20260805 23:13:45.017138 9999 raft_consensus.cc:348] T 00000000000000000000000000000000 P 242b9070a63b4c049d7c7563d64f021b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "242b9070a63b4c049d7c7563d64f021b" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 34379 } }
I20260805 23:13:45.017647 9999 raft_consensus.cc:729] T 00000000000000000000000000000000 P 242b9070a63b4c049d7c7563d64f021b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242b9070a63b4c049d7c7563d64f021b, State: Initialized, Role: FOLLOWER
I20260805 23:13:45.018184 9999 consensus_queue.cc:260] T 00000000000000000000000000000000 P 242b9070a63b4c049d7c7563d64f021b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "242b9070a63b4c049d7c7563d64f021b" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 34379 } }
I20260805 23:13:45.018330 9999 raft_consensus.cc:388] T 00000000000000000000000000000000 P 242b9070a63b4c049d7c7563d64f021b [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:13:45.018426 9999 raft_consensus.cc:482] T 00000000000000000000000000000000 P 242b9070a63b4c049d7c7563d64f021b [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:13:45.018532 9999 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 242b9070a63b4c049d7c7563d64f021b [term 1 FOLLOWER]: Advancing to term 2
I20260805 23:13:45.020486 9999 raft_consensus.cc:504] T 00000000000000000000000000000000 P 242b9070a63b4c049d7c7563d64f021b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "242b9070a63b4c049d7c7563d64f021b" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 34379 } }
I20260805 23:13:45.020835 9999 leader_election.cc:302] T 00000000000000000000000000000000 P 242b9070a63b4c049d7c7563d64f021b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 242b9070a63b4c049d7c7563d64f021b; no voters:
I20260805 23:13:45.021167 9999 leader_election.cc:288] T 00000000000000000000000000000000 P 242b9070a63b4c049d7c7563d64f021b [CANDIDATE]: Term 2 election: Requested vote from peers
I20260805 23:13:45.021337 10004 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 242b9070a63b4c049d7c7563d64f021b [term 2 FOLLOWER]: Leader election won for term 2
I20260805 23:13:45.021622 10004 raft_consensus.cc:686] T 00000000000000000000000000000000 P 242b9070a63b4c049d7c7563d64f021b [term 2 LEADER]: Becoming Leader. State: Replica: 242b9070a63b4c049d7c7563d64f021b, State: Running, Role: LEADER
I20260805 23:13:45.021997 10004 consensus_queue.cc:237] T 00000000000000000000000000000000 P 242b9070a63b4c049d7c7563d64f021b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "242b9070a63b4c049d7c7563d64f021b" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 34379 } }
I20260805 23:13:45.022405 9999 sys_catalog.cc:564] T 00000000000000000000000000000000 P 242b9070a63b4c049d7c7563d64f021b [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:13:45.023000 10004 sys_catalog.cc:455] T 00000000000000000000000000000000 P 242b9070a63b4c049d7c7563d64f021b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 242b9070a63b4c049d7c7563d64f021b. Latest consensus state: current_term: 2 leader_uuid: "242b9070a63b4c049d7c7563d64f021b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "242b9070a63b4c049d7c7563d64f021b" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 34379 } } }
I20260805 23:13:45.023209 10004 sys_catalog.cc:458] T 00000000000000000000000000000000 P 242b9070a63b4c049d7c7563d64f021b [sys.catalog]: This master's current role is: LEADER
I20260805 23:13:45.023412 10005 sys_catalog.cc:455] T 00000000000000000000000000000000 P 242b9070a63b4c049d7c7563d64f021b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "242b9070a63b4c049d7c7563d64f021b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "242b9070a63b4c049d7c7563d64f021b" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 34379 } } }
I20260805 23:13:45.023522 10005 sys_catalog.cc:458] T 00000000000000000000000000000000 P 242b9070a63b4c049d7c7563d64f021b [sys.catalog]: This master's current role is: LEADER
I20260805 23:13:45.023663 10012 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:13:45.028460 10012 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:13:45.029223 10012 catalog_manager.cc:1259] Loaded cluster ID: bd8331ed9dd64faca326013e7f711783
I20260805 23:13:45.029297 10012 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:13:45.032014 10012 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:13:45.032716 10012 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 242b9070a63b4c049d7c7563d64f021b: Loaded TSK: 0
I20260805 23:13:45.050854 10012 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 242b9070a63b4c049d7c7563d64f021b: Generated new TSK 1
I20260805 23:13:45.051141 10012 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260805 23:13:45.105127 10002 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:13:45.105374 10002 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:13:45.105417 10002 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:13:45.109501 10002 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:13:45.109597 10002 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:13:45.109750 10002 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.92.193
I20260805 23:13:45.115051 10002 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:33407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.193.keytab
--principal=kudu/127.25.92.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.92.193:45305
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.92.193
--webserver_port=43515
--tserver_master_addrs=127.25.92.254:34379
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10002
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.92.193
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:13:45.116175 10002 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:13:45.117190 10002 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:13:45.127903 10029 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:13:45.127939 10028 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:13:45.127939 10031 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:13:45.128224 10002 server_base.cc:1034] running on GCE node
I20260805 23:13:45.128723 10002 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:13:45.129413 10002 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:13:45.130657 10002 hybrid_clock.cc:648] HybridClock initialized: now 1785971625130628 us; error 46 us; skew 500 ppm
Aug 05 23:13:45 dist-test-slave-ss4n krb5kdc[9361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971625, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:13:45.133713 10002 init.cc:377] Logged in from keytab as kudu/127.25.92.193@KRBTEST.COM (short username kudu)
I20260805 23:13:45.134760 10002 webserver.cc:460] Webserver started at http://127.25.92.193:43515/ using document root <none> and password file <none>
I20260805 23:13:45.135334 10002 fs_manager.cc:359] Metadata directory not provided
I20260805 23:13:45.135442 10002 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:13:45.139642 10002 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260805 23:13:45.142096 10038 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:13:45.143496 10002 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:13:45.143642 10002 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-0/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-0/wal
uuid: "85d51d37bc304498873dd204279742da"
format_stamp: "Formatted at 2026-08-05 23:13:24 on dist-test-slave-ss4n"
server_key: "b2eecfd72b6f80b9d7e2d72f01a973d8"
server_key_iv: "f4d7aa860e2f69542f8b62006293d9a9"
server_key_version: "encryptionkey@0"
I20260805 23:13:45.144204 10002 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:13:45.163328 10002 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:13:45.166841 10002 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:13:45.167060 10002 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:13:45.167707 10002 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:13:45.168838 10046 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260805 23:13:45.173046 10002 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260805 23:13:45.173115 10002 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.000s sys 0.000s
I20260805 23:13:45.173189 10002 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260805 23:13:45.174921 10002 ts_tablet_manager.cc:610] Registered 1 tablets
I20260805 23:13:45.174966 10002 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260805 23:13:45.175335 10046 tablet_bootstrap.cc:492] T 084470555dc148c48d79b194e2e60621 P 85d51d37bc304498873dd204279742da: Bootstrap starting.
I20260805 23:13:45.185969 10002 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.193:45305
I20260805 23:13:45.186081 10156 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.193:45305 every 8 connection(s)
I20260805 23:13:45.186985 10002 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-0/data/info.pb
I20260805 23:13:45.191039 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 10002
I20260805 23:13:45.191573 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.92.194:44511
--local_ip_for_outbound_sockets=127.25.92.194
--tserver_master_addrs=127.25.92.254:34379
--webserver_port=44771
--webserver_interface=127.25.92.194
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.194.keytab
--principal=kudu/127.25.92.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:33407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260805 23:13:45.194154 10046 log.cc:824] T 084470555dc148c48d79b194e2e60621 P 85d51d37bc304498873dd204279742da: Log is configured to *not* fsync() on all Append() calls
Aug 05 23:13:45 dist-test-slave-ss4n krb5kdc[9361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971625, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.193@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM
I20260805 23:13:45.203965 10046 tablet_bootstrap.cc:492] T 084470555dc148c48d79b194e2e60621 P 85d51d37bc304498873dd204279742da: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260805 23:13:45.204494 10046 tablet_bootstrap.cc:492] T 084470555dc148c48d79b194e2e60621 P 85d51d37bc304498873dd204279742da: Bootstrap complete.
I20260805 23:13:45.204852 10160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:45.189113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.193:48677 (local address 127.25.92.254:34379)
0805 23:13:45.189612 (+ 499us) server_negotiation.cc:206] Beginning negotiation
0805 23:13:45.189630 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0805 23:13:45.189707 (+ 77us) server_negotiation.cc:407] Connection header received
0805 23:13:45.191385 (+ 1678us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:13:45.191404 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:13:45.191823 (+ 419us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:13:45.192180 (+ 357us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:13:45.193589 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:13:45.195147 (+ 1558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:13:45.196537 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:13:45.197005 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:13:45.197201 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:13:45.199713 (+ 2512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:13:45.199719 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:13:45.199731 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:13:45.199758 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:13:45.201611 (+ 1853us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:13:45.202152 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:13:45.202164 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:13:45.202167 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:13:45.202299 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:13:45.202656 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:13:45.202660 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:13:45.202662 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:13:45.203001 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:13:45.203153 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0805 23:13:45.203664 (+ 511us) server_negotiation.cc:299] Negotiation successful
0805 23:13:45.203938 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":114,"threads_started":1}
I20260805 23:13:45.204864 10159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:45.188947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:34379 (local address 127.25.92.193:48677)
0805 23:13:45.189547 (+ 600us) negotiation.cc:107] Waiting for socket to connect
0805 23:13:45.189605 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0805 23:13:45.190877 (+ 1272us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:13:45.192279 (+ 1402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:13:45.192289 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:13:45.192762 (+ 473us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:13:45.193364 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:13:45.193383 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:13:45.195326 (+ 1943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:13:45.195329 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:13:45.196408 (+ 1079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:13:45.196419 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:13:45.197162 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:13:45.197166 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:13:45.197348 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:13:45.198029 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:13:45.198046 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:13:45.199503 (+ 1457us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:13:45.201734 (+ 2231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:13:45.201740 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:13:45.201753 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:13:45.202020 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:13:45.202408 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:13:45.202411 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:13:45.202413 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:13:45.202553 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:13:45.203132 (+ 579us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:13:45.203137 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:13:45.203366 (+ 229us) client_negotiation.cc:769] Sending connection context
0805 23:13:45.203548 (+ 182us) client_negotiation.cc:240] Negotiation successful
0805 23:13:45.203961 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":152,"threads_started":1}
I20260805 23:13:45.206025 10046 ts_tablet_manager.cc:1397] T 084470555dc148c48d79b194e2e60621 P 85d51d37bc304498873dd204279742da: Time spent bootstrapping tablet: real 0.031s user 0.022s sys 0.008s
I20260805 23:13:45.207371 10157 heartbeater.cc:347] Connected to a master server at 127.25.92.254:34379
I20260805 23:13:45.207608 10157 heartbeater.cc:464] Registering TS with master...
I20260805 23:13:45.208077 10157 heartbeater.cc:511] Master 127.25.92.254:34379 requested a full tablet report, sending...
I20260805 23:13:45.208910 10046 raft_consensus.cc:348] T 084470555dc148c48d79b194e2e60621 P 85d51d37bc304498873dd204279742da [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69b377dded7945329549bf5a64bf87b1" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 44511 } } peers { permanent_uuid: "fd89afa0150a4437b4f2ea95593ccaf0" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 41629 } } peers { permanent_uuid: "85d51d37bc304498873dd204279742da" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 45305 } }
I20260805 23:13:45.209460 10046 raft_consensus.cc:729] T 084470555dc148c48d79b194e2e60621 P 85d51d37bc304498873dd204279742da [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85d51d37bc304498873dd204279742da, State: Initialized, Role: FOLLOWER
I20260805 23:13:45.209996 10046 consensus_queue.cc:260] T 084470555dc148c48d79b194e2e60621 P 85d51d37bc304498873dd204279742da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69b377dded7945329549bf5a64bf87b1" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 44511 } } peers { permanent_uuid: "fd89afa0150a4437b4f2ea95593ccaf0" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 41629 } } peers { permanent_uuid: "85d51d37bc304498873dd204279742da" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 45305 } }
I20260805 23:13:45.210554 9963 ts_manager.cc:194] Registered new tserver with Master: 85d51d37bc304498873dd204279742da (127.25.92.193:45305)
I20260805 23:13:45.211009 10046 ts_tablet_manager.cc:1428] T 084470555dc148c48d79b194e2e60621 P 85d51d37bc304498873dd204279742da: Time spent starting tablet: real 0.005s user 0.001s sys 0.004s
I20260805 23:13:45.214184 9963 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.92.193@KRBTEST.COM'} at 127.25.92.193:48677
I20260805 23:13:45.216655 10157 heartbeater.cc:503] Master 127.25.92.254:34379 was elected leader, sending a full tablet report...
Aug 05 23:13:45 dist-test-slave-ss4n krb5kdc[9361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971624, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.254@KRBTEST.COM for kudu/127.25.92.193@KRBTEST.COM
I20260805 23:13:45.225364 10165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:45.213364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47372 (local address 127.25.92.193:45305)
0805 23:13:45.214981 (+ 1617us) server_negotiation.cc:206] Beginning negotiation
0805 23:13:45.214990 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0805 23:13:45.215010 (+ 20us) server_negotiation.cc:407] Connection header received
0805 23:13:45.215065 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:13:45.215069 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:13:45.215284 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:13:45.215421 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:13:45.216184 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:13:45.217198 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:13:45.218436 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:13:45.218765 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:13:45.218850 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:13:45.221362 (+ 2512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:13:45.221373 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:13:45.221386 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:13:45.221413 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:13:45.223252 (+ 1839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:13:45.223862 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:13:45.223868 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:13:45.223871 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:13:45.223969 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:13:45.224308 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:13:45.224311 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:13:45.224312 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:13:45.224592 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:13:45.224709 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0805 23:13:45.225089 (+ 380us) server_negotiation.cc:299] Negotiation successful
0805 23:13:45.225233 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1520,"thread_start_us":111,"threads_started":1}
I20260805 23:13:45.225417 10164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:45.213261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.193:45305 (local address 127.0.0.1:47372)
0805 23:13:45.213745 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0805 23:13:45.213798 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0805 23:13:45.213925 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:13:45.215453 (+ 1528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:13:45.215459 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:13:45.215894 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:13:45.216039 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:13:45.216047 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:13:45.217326 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:13:45.217330 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:13:45.218310 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:13:45.218322 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:13:45.218890 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:13:45.218894 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:13:45.219006 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:13:45.219578 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:13:45.219593 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:13:45.221199 (+ 1606us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:13:45.223378 (+ 2179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:13:45.223396 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:13:45.223401 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:13:45.223734 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:13:45.224089 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:13:45.224092 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:13:45.224094 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:13:45.224205 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:13:45.224720 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:13:45.224726 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:13:45.224858 (+ 132us) client_negotiation.cc:769] Sending connection context
0805 23:13:45.225012 (+ 154us) client_negotiation.cc:240] Negotiation successful
0805 23:13:45.225241 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":139,"threads_started":1}
I20260805 23:13:45.226922 10089 tablet_service.cc:1514] Processing DeleteTablet for tablet 084470555dc148c48d79b194e2e60621 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:13:24 UTC) from {username='kudu', principal='kudu/127.25.92.254@KRBTEST.COM'} at 127.0.0.1:47372
I20260805 23:13:45.227787 10166 tablet_replica.cc:330] stopping tablet replica
I20260805 23:13:45.228096 10166 raft_consensus.cc:2229] T 084470555dc148c48d79b194e2e60621 P 85d51d37bc304498873dd204279742da [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 23:13:45.228348 10166 raft_consensus.cc:2258] T 084470555dc148c48d79b194e2e60621 P 85d51d37bc304498873dd204279742da [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:13:45.229312 10166 ts_tablet_manager.cc:1905] T 084470555dc148c48d79b194e2e60621 P 85d51d37bc304498873dd204279742da: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:13:45.233137 10166 ts_tablet_manager.cc:1918] T 084470555dc148c48d79b194e2e60621 P 85d51d37bc304498873dd204279742da: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 23:13:45.233237 10166 log.cc:1196] T 084470555dc148c48d79b194e2e60621 P 85d51d37bc304498873dd204279742da: Deleting WAL directory at /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-0/wal/wals/084470555dc148c48d79b194e2e60621
I20260805 23:13:45.233654 10166 ts_tablet_manager.cc:1939] T 084470555dc148c48d79b194e2e60621 P 85d51d37bc304498873dd204279742da: Deleting consensus metadata
I20260805 23:13:45.235514 9947 catalog_manager.cc:4841] TS 85d51d37bc304498873dd204279742da (127.25.92.193:45305): tablet 084470555dc148c48d79b194e2e60621 (table test-table [id=0c49d00738f14a6eaee90131887926d0]) successfully deleted
W20260805 23:13:45.313647 10161 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:13:45.313917 10161 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:13:45.313963 10161 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:13:45.317917 10161 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:13:45.317981 10161 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:13:45.318041 10161 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.92.194
I20260805 23:13:45.323338 10161 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:33407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.194.keytab
--principal=kudu/127.25.92.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.92.194:44511
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.92.194
--webserver_port=44771
--tserver_master_addrs=127.25.92.254:34379
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10161
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.92.194
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:13:45.324491 10161 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:13:45.325497 10161 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:13:45.335335 10172 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:13:45.335383 10161 server_base.cc:1034] running on GCE node
W20260805 23:13:45.335382 10171 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:13:45.335548 10174 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:13:45.336061 10161 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:13:45.336757 10161 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:13:45.337957 10161 hybrid_clock.cc:648] HybridClock initialized: now 1785971625337931 us; error 55 us; skew 500 ppm
Aug 05 23:13:45 dist-test-slave-ss4n krb5kdc[9361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971625, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:13:45.340967 10161 init.cc:377] Logged in from keytab as kudu/127.25.92.194@KRBTEST.COM (short username kudu)
I20260805 23:13:45.342108 10161 webserver.cc:460] Webserver started at http://127.25.92.194:44771/ using document root <none> and password file <none>
I20260805 23:13:45.342808 10161 fs_manager.cc:359] Metadata directory not provided
I20260805 23:13:45.342942 10161 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:13:45.347211 10161 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260805 23:13:45.349843 10181 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:13:45.351197 10161 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260805 23:13:45.351341 10161 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-1/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-1/wal
uuid: "69b377dded7945329549bf5a64bf87b1"
format_stamp: "Formatted at 2026-08-05 23:13:24 on dist-test-slave-ss4n"
server_key: "a6fb1dbdbb49b2b0b0e8f2fdd7d0fd6d"
server_key_iv: "38dbd67ee19f383f3a91fe3b8aafd53e"
server_key_version: "encryptionkey@0"
I20260805 23:13:45.351930 10161 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:13:45.367542 10161 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:13:45.371395 10161 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:13:45.371621 10161 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:13:45.372254 10161 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:13:45.373227 10161 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:13:45.373317 10161 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:13:45.373390 10161 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:13:45.373441 10161 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:13:45.382742 10161 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.194:44511
I20260805 23:13:45.382836 10297 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.194:44511 every 8 connection(s)
I20260805 23:13:45.383870 10161 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-1/data/info.pb
I20260805 23:13:45.389533 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 10161
I20260805 23:13:45.389995 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.92.195:41629
--local_ip_for_outbound_sockets=127.25.92.195
--tserver_master_addrs=127.25.92.254:34379
--webserver_port=39537
--webserver_interface=127.25.92.195
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.195.keytab
--principal=kudu/127.25.92.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:33407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 05 23:13:45 dist-test-slave-ss4n krb5kdc[9361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971625, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.194@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM
I20260805 23:13:45.398466 10160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:45.385972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.194:58007 (local address 127.25.92.254:34379)
0805 23:13:45.386161 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0805 23:13:45.386167 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:13:45.387020 (+ 853us) server_negotiation.cc:407] Connection header received
0805 23:13:45.388231 (+ 1211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:13:45.388233 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:13:45.388282 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:13:45.388358 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:13:45.389545 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:13:45.390277 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:13:45.391456 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:13:45.391712 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:13:45.391829 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:13:45.394513 (+ 2684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:13:45.394517 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:13:45.394520 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:13:45.394555 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:13:45.396304 (+ 1749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:13:45.396849 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:13:45.396852 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:13:45.396854 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:13:45.396941 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:13:45.397293 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:13:45.397297 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:13:45.397299 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:13:45.397548 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:13:45.397650 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0805 23:13:45.398144 (+ 494us) server_negotiation.cc:299] Negotiation successful
0805 23:13:45.398286 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260805 23:13:45.399294 10300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:45.386389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:34379 (local address 127.25.92.194:58007)
0805 23:13:45.386866 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0805 23:13:45.386909 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0805 23:13:45.388029 (+ 1120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:13:45.388485 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:13:45.388491 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:13:45.388852 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:13:45.389347 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:13:45.389362 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:13:45.390424 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:13:45.390430 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:13:45.391315 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:13:45.391325 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:13:45.391874 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:13:45.391878 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:13:45.392053 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:13:45.392720 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:13:45.392736 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:13:45.394329 (+ 1593us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:13:45.396430 (+ 2101us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:13:45.396436 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:13:45.396453 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:13:45.396736 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:13:45.397041 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:13:45.397045 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:13:45.397047 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:13:45.397194 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:13:45.397646 (+ 452us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:13:45.397652 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:13:45.397905 (+ 253us) client_negotiation.cc:769] Sending connection context
0805 23:13:45.398092 (+ 187us) client_negotiation.cc:240] Negotiation successful
0805 23:13:45.398431 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":121,"threads_started":1}
I20260805 23:13:45.400533 10298 heartbeater.cc:347] Connected to a master server at 127.25.92.254:34379
I20260805 23:13:45.400825 10298 heartbeater.cc:464] Registering TS with master...
I20260805 23:13:45.401391 10298 heartbeater.cc:511] Master 127.25.92.254:34379 requested a full tablet report, sending...
I20260805 23:13:45.402570 9963 ts_manager.cc:194] Registered new tserver with Master: 69b377dded7945329549bf5a64bf87b1 (127.25.92.194:44511)
I20260805 23:13:45.403316 9963 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.92.194@KRBTEST.COM'} at 127.25.92.194:58007
W20260805 23:13:45.504441 10301 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:13:45.504722 10301 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:13:45.504819 10301 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:13:45.508919 10301 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:13:45.508993 10301 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:13:45.509053 10301 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.92.195
I20260805 23:13:45.514272 10301 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:33407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.195.keytab
--principal=kudu/127.25.92.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.92.195:41629
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.92.195
--webserver_port=39537
--tserver_master_addrs=127.25.92.254:34379
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10301
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.92.195
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:13:45.515368 10301 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:13:45.516425 10301 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:13:45.526255 10306 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:13:45.526335 10309 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:13:45.526255 10307 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:13:45.526564 10301 server_base.cc:1034] running on GCE node
I20260805 23:13:45.527117 10301 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:13:45.527731 10301 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:13:45.528944 10301 hybrid_clock.cc:648] HybridClock initialized: now 1785971625528909 us; error 44 us; skew 500 ppm
Aug 05 23:13:45 dist-test-slave-ss4n krb5kdc[9361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971625, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:13:45.531788 10301 init.cc:377] Logged in from keytab as kudu/127.25.92.195@KRBTEST.COM (short username kudu)
I20260805 23:13:45.532900 10301 webserver.cc:460] Webserver started at http://127.25.92.195:39537/ using document root <none> and password file <none>
I20260805 23:13:45.533457 10301 fs_manager.cc:359] Metadata directory not provided
I20260805 23:13:45.533609 10301 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:13:45.537433 10301 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.002s
I20260805 23:13:45.539637 10316 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:13:45.540848 10301 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260805 23:13:45.540948 10301 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-2/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-2/wal
uuid: "fd89afa0150a4437b4f2ea95593ccaf0"
format_stamp: "Formatted at 2026-08-05 23:13:24 on dist-test-slave-ss4n"
server_key: "dad15bf071517ab654fa7c853f5edbfb"
server_key_iv: "3cf942aaac65f2be89eed7fbce6f0497"
server_key_version: "encryptionkey@0"
I20260805 23:13:45.541584 10301 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:13:45.558416 10301 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:13:45.561933 10301 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:13:45.562160 10301 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:13:45.562789 10301 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:13:45.563657 10301 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:13:45.563728 10301 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:13:45.563818 10301 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:13:45.563869 10301 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:13:45.572793 10301 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.195:41629
I20260805 23:13:45.572861 10431 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.195:41629 every 8 connection(s)
I20260805 23:13:45.573953 10301 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-2/data/info.pb
I20260805 23:13:45.577080 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 10301
Aug 05 23:13:45 dist-test-slave-ss4n krb5kdc[9361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971625, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.195@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM
I20260805 23:13:45.587829 10437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:45.577936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38570 (local address 127.25.92.254:34379)
0805 23:13:45.578547 (+ 611us) server_negotiation.cc:206] Beginning negotiation
0805 23:13:45.578552 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:13:45.578565 (+ 13us) server_negotiation.cc:407] Connection header received
0805 23:13:45.578612 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:13:45.578615 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:13:45.578677 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:13:45.578770 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:13:45.579806 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:13:45.580590 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:13:45.581831 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:13:45.582172 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:13:45.582295 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:13:45.584091 (+ 1796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:13:45.584095 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:13:45.584100 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:13:45.584142 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:13:45.585901 (+ 1759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:13:45.586446 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:13:45.586452 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:13:45.586454 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:13:45.586597 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:13:45.586900 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:13:45.586904 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:13:45.586906 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:13:45.587097 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:13:45.587351 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0805 23:13:45.587516 (+ 165us) server_negotiation.cc:299] Negotiation successful
0805 23:13:45.587639 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":537,"thread_start_us":89,"threads_started":1}
I20260805 23:13:45.589875 10160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:45.576019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.195:42927 (local address 127.25.92.254:34379)
0805 23:13:45.576179 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0805 23:13:45.576183 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:13:45.577204 (+ 1021us) server_negotiation.cc:407] Connection header received
0805 23:13:45.578927 (+ 1723us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:13:45.578934 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:13:45.578997 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:13:45.579098 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:13:45.580469 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:13:45.581309 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:13:45.582781 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:13:45.583116 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:13:45.583204 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:13:45.585875 (+ 2671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:13:45.585879 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:13:45.585882 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:13:45.585913 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:13:45.587388 (+ 1475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:13:45.588093 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:13:45.588096 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:13:45.588097 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:13:45.588175 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:13:45.588608 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:13:45.588611 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:13:45.588613 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:13:45.588801 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:13:45.588879 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0805 23:13:45.589605 (+ 726us) server_negotiation.cc:299] Negotiation successful
0805 23:13:45.589721 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260805 23:13:45.590843 10434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:45.576500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:34379 (local address 127.25.92.195:42927)
0805 23:13:45.577002 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0805 23:13:45.577063 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0805 23:13:45.578695 (+ 1632us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:13:45.579286 (+ 591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:13:45.579293 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:13:45.579646 (+ 353us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:13:45.580306 (+ 660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:13:45.580324 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:13:45.581443 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:13:45.581445 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:13:45.582659 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:13:45.582669 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:13:45.583292 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:13:45.583298 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:13:45.583539 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:13:45.584172 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:13:45.584188 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:13:45.585710 (+ 1522us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:13:45.587522 (+ 1812us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:13:45.587529 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:13:45.587546 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:13:45.587934 (+ 388us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:13:45.588272 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:13:45.588275 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:13:45.588277 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:13:45.588492 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:13:45.588919 (+ 427us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:13:45.588927 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:13:45.589298 (+ 371us) client_negotiation.cc:769] Sending connection context
0805 23:13:45.589464 (+ 166us) client_negotiation.cc:240] Negotiation successful
0805 23:13:45.589815 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":107,"threads_started":1}
I20260805 23:13:45.592023 10432 heartbeater.cc:347] Connected to a master server at 127.25.92.254:34379
I20260805 23:13:45.592271 10432 heartbeater.cc:464] Registering TS with master...
I20260805 23:13:45.592769 10432 heartbeater.cc:511] Master 127.25.92.254:34379 requested a full tablet report, sending...
I20260805 23:13:45.593909 9963 ts_manager.cc:194] Registered new tserver with Master: fd89afa0150a4437b4f2ea95593ccaf0 (127.25.92.195:41629)
I20260805 23:13:45.594503 9963 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.92.195@KRBTEST.COM'} at 127.25.92.195:42927
I20260805 23:13:45.602486 25971 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260805 23:13:45.606909 10160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:45.603214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38572 (local address 127.25.92.254:34379)
0805 23:13:45.603368 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0805 23:13:45.603372 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:13:45.603654 (+ 282us) server_negotiation.cc:407] Connection header received
0805 23:13:45.603828 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:13:45.603831 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:13:45.603934 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:13:45.604031 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 23:13:45.604231 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:13:45.604783 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:13:45.605754 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:13:45.605981 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:13:45.606072 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:13:45.606428 (+ 356us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 23:13:45.606572 (+ 144us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0805 23:13:45.606796 (+ 224us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:38572: token expired
Metrics: {"server-negotiator.queue_time_us":71}
W20260805 23:13:45.607008 10160 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:38572: token expired
W20260805 23:13:45.607384 9886 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.25.92.254:34379)
I20260805 23:13:45.615576 10160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:45.607783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38576 (local address 127.25.92.254:34379)
0805 23:13:45.608013 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0805 23:13:45.608019 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:13:45.608033 (+ 14us) server_negotiation.cc:407] Connection header received
0805 23:13:45.608193 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:13:45.608196 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:13:45.608244 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:13:45.608325 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:13:45.608959 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:13:45.609465 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:13:45.610342 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:13:45.610580 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:13:45.610690 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:13:45.612166 (+ 1476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:13:45.612170 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:13:45.612172 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:13:45.612199 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:13:45.613764 (+ 1565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:13:45.614227 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:13:45.614231 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:13:45.614233 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:13:45.614320 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:13:45.614665 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:13:45.614670 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:13:45.614672 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:13:45.614855 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:13:45.614998 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0805 23:13:45.615281 (+ 283us) server_negotiation.cc:299] Negotiation successful
0805 23:13:45.615404 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149}
I20260805 23:13:45.618484 9963 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38576:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260805 23:13:45.621990 9963 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260805 23:13:45.630425 10089 tablet_service.cc:1467] Processing CreateTablet for tablet 748bd38edc9844d3941a5797032ce70d (DEFAULT_TABLE table=test-table [id=e2997c9f521a4d7f833d49c342d63546]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:13:45.631145 10089 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 748bd38edc9844d3941a5797032ce70d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:13:45.634225 10046 tablet_bootstrap.cc:492] T 748bd38edc9844d3941a5797032ce70d P 85d51d37bc304498873dd204279742da: Bootstrap starting.
I20260805 23:13:45.635198 10046 tablet_bootstrap.cc:654] T 748bd38edc9844d3941a5797032ce70d P 85d51d37bc304498873dd204279742da: Neither blocks nor log segments found. Creating new log.
I20260805 23:13:45.635913 10164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:45.629385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.195:41629 (local address 127.0.0.1:41700)
0805 23:13:45.629711 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0805 23:13:45.629730 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0805 23:13:45.629815 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:13:45.631090 (+ 1275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:13:45.631093 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:13:45.631123 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:13:45.631196 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:13:45.631206 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:13:45.633045 (+ 1839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:13:45.633052 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:13:45.634032 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:13:45.634043 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:13:45.635321 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:13:45.635325 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:13:45.635462 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:13:45.635545 (+ 83us) client_negotiation.cc:769] Sending connection context
0805 23:13:45.635626 (+ 81us) client_negotiation.cc:240] Negotiation successful
0805 23:13:45.635738 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"mutex_wait_us":30}
I20260805 23:13:45.636103 10445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:45.630035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41238 (local address 127.25.92.194:44511)
0805 23:13:45.630418 (+ 383us) server_negotiation.cc:206] Beginning negotiation
0805 23:13:45.630431 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0805 23:13:45.630458 (+ 27us) server_negotiation.cc:407] Connection header received
0805 23:13:45.630554 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:13:45.630559 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:13:45.630781 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:13:45.630956 (+ 175us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:13:45.631316 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:13:45.632619 (+ 1303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:13:45.634258 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:13:45.635498 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:13:45.635625 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:13:45.635775 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0805 23:13:45.635842 (+ 67us) server_negotiation.cc:299] Negotiation successful
0805 23:13:45.635944 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":169,"threads_started":1}
I20260805 23:13:45.636415 10444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:45.629578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.194:44511 (local address 127.0.0.1:41238)
0805 23:13:45.630166 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0805 23:13:45.630184 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0805 23:13:45.630284 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:13:45.630964 (+ 680us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:13:45.630971 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:13:45.630996 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:13:45.631162 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:13:45.631170 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:13:45.632746 (+ 1576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:13:45.632752 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:13:45.634103 (+ 1351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:13:45.634116 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:13:45.635627 (+ 1511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:13:45.635631 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:13:45.635719 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:13:45.635741 (+ 22us) client_negotiation.cc:769] Sending connection context
0805 23:13:45.635800 (+ 59us) client_negotiation.cc:240] Negotiation successful
0805 23:13:45.635876 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"mutex_wait_us":26,"thread_start_us":129,"threads_started":1}
I20260805 23:13:45.636435 10442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:45.629582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41700 (local address 127.25.92.195:41629)
0805 23:13:45.630534 (+ 952us) server_negotiation.cc:206] Beginning negotiation
0805 23:13:45.630547 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0805 23:13:45.630571 (+ 24us) server_negotiation.cc:407] Connection header received
0805 23:13:45.630668 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:13:45.630673 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:13:45.630875 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:13:45.631358 (+ 483us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:13:45.631531 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:13:45.632920 (+ 1389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:13:45.634221 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:13:45.635155 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:13:45.636005 (+ 850us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:13:45.636146 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0805 23:13:45.636199 (+ 53us) server_negotiation.cc:299] Negotiation successful
0805 23:13:45.636297 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":857,"thread_start_us":139,"threads_started":1}
I20260805 23:13:45.638154 10046 tablet_bootstrap.cc:492] T 748bd38edc9844d3941a5797032ce70d P 85d51d37bc304498873dd204279742da: No bootstrap required, opened a new log
I20260805 23:13:45.638269 10046 ts_tablet_manager.cc:1397] T 748bd38edc9844d3941a5797032ce70d P 85d51d37bc304498873dd204279742da: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.001s
I20260805 23:13:45.638834 10046 raft_consensus.cc:348] T 748bd38edc9844d3941a5797032ce70d P 85d51d37bc304498873dd204279742da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd89afa0150a4437b4f2ea95593ccaf0" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 41629 } } peers { permanent_uuid: "69b377dded7945329549bf5a64bf87b1" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 44511 } } peers { permanent_uuid: "85d51d37bc304498873dd204279742da" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 45305 } }
I20260805 23:13:45.638978 10046 raft_consensus.cc:374] T 748bd38edc9844d3941a5797032ce70d P 85d51d37bc304498873dd204279742da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:13:45.639031 10046 raft_consensus.cc:729] T 748bd38edc9844d3941a5797032ce70d P 85d51d37bc304498873dd204279742da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85d51d37bc304498873dd204279742da, State: Initialized, Role: FOLLOWER
I20260805 23:13:45.639166 10046 consensus_queue.cc:260] T 748bd38edc9844d3941a5797032ce70d P 85d51d37bc304498873dd204279742da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd89afa0150a4437b4f2ea95593ccaf0" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 41629 } } peers { permanent_uuid: "69b377dded7945329549bf5a64bf87b1" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 44511 } } peers { permanent_uuid: "85d51d37bc304498873dd204279742da" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 45305 } }
I20260805 23:13:45.639474 10046 ts_tablet_manager.cc:1428] T 748bd38edc9844d3941a5797032ce70d P 85d51d37bc304498873dd204279742da: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260805 23:13:45.639698 10230 tablet_service.cc:1467] Processing CreateTablet for tablet 748bd38edc9844d3941a5797032ce70d (DEFAULT_TABLE table=test-table [id=e2997c9f521a4d7f833d49c342d63546]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:13:45.639699 10365 tablet_service.cc:1467] Processing CreateTablet for tablet 748bd38edc9844d3941a5797032ce70d (DEFAULT_TABLE table=test-table [id=e2997c9f521a4d7f833d49c342d63546]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:13:45.640844 10365 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 748bd38edc9844d3941a5797032ce70d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:13:45.640838 10230 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 748bd38edc9844d3941a5797032ce70d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:13:45.649431 10448 tablet_bootstrap.cc:492] T 748bd38edc9844d3941a5797032ce70d P fd89afa0150a4437b4f2ea95593ccaf0: Bootstrap starting.
I20260805 23:13:45.649443 10447 tablet_bootstrap.cc:492] T 748bd38edc9844d3941a5797032ce70d P 69b377dded7945329549bf5a64bf87b1: Bootstrap starting.
I20260805 23:13:45.652436 10447 tablet_bootstrap.cc:654] T 748bd38edc9844d3941a5797032ce70d P 69b377dded7945329549bf5a64bf87b1: Neither blocks nor log segments found. Creating new log.
I20260805 23:13:45.652456 10448 tablet_bootstrap.cc:654] T 748bd38edc9844d3941a5797032ce70d P fd89afa0150a4437b4f2ea95593ccaf0: Neither blocks nor log segments found. Creating new log.
I20260805 23:13:45.653390 10448 log.cc:824] T 748bd38edc9844d3941a5797032ce70d P fd89afa0150a4437b4f2ea95593ccaf0: Log is configured to *not* fsync() on all Append() calls
I20260805 23:13:45.653391 10447 log.cc:824] T 748bd38edc9844d3941a5797032ce70d P 69b377dded7945329549bf5a64bf87b1: Log is configured to *not* fsync() on all Append() calls
I20260805 23:13:45.655268 10447 tablet_bootstrap.cc:492] T 748bd38edc9844d3941a5797032ce70d P 69b377dded7945329549bf5a64bf87b1: No bootstrap required, opened a new log
I20260805 23:13:45.655416 10448 tablet_bootstrap.cc:492] T 748bd38edc9844d3941a5797032ce70d P fd89afa0150a4437b4f2ea95593ccaf0: No bootstrap required, opened a new log
I20260805 23:13:45.655535 10447 ts_tablet_manager.cc:1397] T 748bd38edc9844d3941a5797032ce70d P 69b377dded7945329549bf5a64bf87b1: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.002s
I20260805 23:13:45.655618 10448 ts_tablet_manager.cc:1397] T 748bd38edc9844d3941a5797032ce70d P fd89afa0150a4437b4f2ea95593ccaf0: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.002s
I20260805 23:13:45.658583 10448 raft_consensus.cc:348] T 748bd38edc9844d3941a5797032ce70d P fd89afa0150a4437b4f2ea95593ccaf0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd89afa0150a4437b4f2ea95593ccaf0" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 41629 } } peers { permanent_uuid: "69b377dded7945329549bf5a64bf87b1" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 44511 } } peers { permanent_uuid: "85d51d37bc304498873dd204279742da" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 45305 } }
I20260805 23:13:45.658586 10447 raft_consensus.cc:348] T 748bd38edc9844d3941a5797032ce70d P 69b377dded7945329549bf5a64bf87b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd89afa0150a4437b4f2ea95593ccaf0" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 41629 } } peers { permanent_uuid: "69b377dded7945329549bf5a64bf87b1" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 44511 } } peers { permanent_uuid: "85d51d37bc304498873dd204279742da" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 45305 } }
I20260805 23:13:45.658841 10448 raft_consensus.cc:374] T 748bd38edc9844d3941a5797032ce70d P fd89afa0150a4437b4f2ea95593ccaf0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:13:45.658835 10447 raft_consensus.cc:374] T 748bd38edc9844d3941a5797032ce70d P 69b377dded7945329549bf5a64bf87b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:13:45.658921 10447 raft_consensus.cc:729] T 748bd38edc9844d3941a5797032ce70d P 69b377dded7945329549bf5a64bf87b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69b377dded7945329549bf5a64bf87b1, State: Initialized, Role: FOLLOWER
I20260805 23:13:45.658921 10448 raft_consensus.cc:729] T 748bd38edc9844d3941a5797032ce70d P fd89afa0150a4437b4f2ea95593ccaf0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd89afa0150a4437b4f2ea95593ccaf0, State: Initialized, Role: FOLLOWER
I20260805 23:13:45.659394 10448 consensus_queue.cc:260] T 748bd38edc9844d3941a5797032ce70d P fd89afa0150a4437b4f2ea95593ccaf0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd89afa0150a4437b4f2ea95593ccaf0" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 41629 } } peers { permanent_uuid: "69b377dded7945329549bf5a64bf87b1" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 44511 } } peers { permanent_uuid: "85d51d37bc304498873dd204279742da" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 45305 } }
I20260805 23:13:45.659394 10447 consensus_queue.cc:260] T 748bd38edc9844d3941a5797032ce70d P 69b377dded7945329549bf5a64bf87b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd89afa0150a4437b4f2ea95593ccaf0" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 41629 } } peers { permanent_uuid: "69b377dded7945329549bf5a64bf87b1" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 44511 } } peers { permanent_uuid: "85d51d37bc304498873dd204279742da" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 45305 } }
I20260805 23:13:45.660744 10448 ts_tablet_manager.cc:1428] T 748bd38edc9844d3941a5797032ce70d P fd89afa0150a4437b4f2ea95593ccaf0: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20260805 23:13:45.660743 10447 ts_tablet_manager.cc:1428] T 748bd38edc9844d3941a5797032ce70d P 69b377dded7945329549bf5a64bf87b1: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20260805 23:13:45.660743 10298 heartbeater.cc:503] Master 127.25.92.254:34379 was elected leader, sending a full tablet report...
I20260805 23:13:45.660750 10432 heartbeater.cc:503] Master 127.25.92.254:34379 was elected leader, sending a full tablet report...
W20260805 23:13:45.688323 10158 tablet.cc:2369] T 748bd38edc9844d3941a5797032ce70d P 85d51d37bc304498873dd204279742da: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260805 23:13:45.824997 10433 tablet.cc:2369] T 748bd38edc9844d3941a5797032ce70d P fd89afa0150a4437b4f2ea95593ccaf0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260805 23:13:45.849387 10163 raft_consensus.cc:482] T 748bd38edc9844d3941a5797032ce70d P 85d51d37bc304498873dd204279742da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 23:13:45.849787 10163 raft_consensus.cc:504] T 748bd38edc9844d3941a5797032ce70d P 85d51d37bc304498873dd204279742da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd89afa0150a4437b4f2ea95593ccaf0" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 41629 } } peers { permanent_uuid: "69b377dded7945329549bf5a64bf87b1" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 44511 } } peers { permanent_uuid: "85d51d37bc304498873dd204279742da" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 45305 } }
I20260805 23:13:45.851209 10163 leader_election.cc:288] T 748bd38edc9844d3941a5797032ce70d P 85d51d37bc304498873dd204279742da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fd89afa0150a4437b4f2ea95593ccaf0 (127.25.92.195:41629), 69b377dded7945329549bf5a64bf87b1 (127.25.92.194:44511)
I20260805 23:13:45.856240 10445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:45.851628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.193:36077 (local address 127.25.92.194:44511)
0805 23:13:45.851837 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0805 23:13:45.851843 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:13:45.851954 (+ 111us) server_negotiation.cc:407] Connection header received
0805 23:13:45.852205 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:13:45.852211 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:13:45.852284 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:13:45.852387 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:13:45.852743 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:13:45.853712 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:13:45.855056 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:13:45.855469 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:13:45.855609 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:13:45.855672 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0805 23:13:45.855978 (+ 306us) server_negotiation.cc:299] Negotiation successful
0805 23:13:45.856073 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112}
I20260805 23:13:45.856405 10454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:45.851486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.195:41629 (local address 127.25.92.193:35975)
0805 23:13:45.852015 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0805 23:13:45.852029 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0805 23:13:45.852124 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:13:45.852580 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:13:45.852583 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:13:45.852598 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:13:45.852689 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:13:45.852698 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:13:45.853453 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:13:45.853460 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:13:45.854722 (+ 1262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:13:45.854735 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:13:45.855903 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:13:45.855908 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:13:45.856012 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:13:45.856036 (+ 24us) client_negotiation.cc:769] Sending connection context
0805 23:13:45.856155 (+ 119us) client_negotiation.cc:240] Negotiation successful
0805 23:13:45.856231 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"mutex_wait_us":40,"thread_start_us":66,"threads_started":1}
I20260805 23:13:45.856463 10442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:45.851873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.193:35975 (local address 127.25.92.195:41629)
0805 23:13:45.852023 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0805 23:13:45.852029 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:13:45.852088 (+ 59us) server_negotiation.cc:407] Connection header received
0805 23:13:45.852320 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:13:45.852324 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:13:45.852386 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:13:45.852468 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:13:45.852794 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:13:45.853312 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:13:45.854860 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:13:45.855752 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:13:45.855941 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:13:45.856021 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0805 23:13:45.856180 (+ 159us) server_negotiation.cc:299] Negotiation successful
0805 23:13:45.856285 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260805 23:13:45.857118 10453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:45.851485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.194:44511 (local address 127.25.92.193:36077)
0805 23:13:45.851841 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0805 23:13:45.851858 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0805 23:13:45.852057 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:13:45.852408 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:13:45.852413 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:13:45.852447 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:13:45.852605 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:13:45.852613 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:13:45.853846 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:13:45.853850 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:13:45.854927 (+ 1077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:13:45.854936 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:13:45.855605 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:13:45.855609 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:13:45.855719 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:13:45.855832 (+ 113us) client_negotiation.cc:769] Sending connection context
0805 23:13:45.856846 (+ 1014us) client_negotiation.cc:240] Negotiation successful
0805 23:13:45.856946 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":94,"threads_started":1}
I20260805 23:13:45.857184 10385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "748bd38edc9844d3941a5797032ce70d" candidate_uuid: "85d51d37bc304498873dd204279742da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd89afa0150a4437b4f2ea95593ccaf0" is_pre_election: true
I20260805 23:13:45.857611 10385 raft_consensus.cc:2454] T 748bd38edc9844d3941a5797032ce70d P fd89afa0150a4437b4f2ea95593ccaf0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85d51d37bc304498873dd204279742da in term 0.
I20260805 23:13:45.858173 10040 leader_election.cc:302] T 748bd38edc9844d3941a5797032ce70d P 85d51d37bc304498873dd204279742da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85d51d37bc304498873dd204279742da, fd89afa0150a4437b4f2ea95593ccaf0; no voters:
I20260805 23:13:45.858502 10250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "748bd38edc9844d3941a5797032ce70d" candidate_uuid: "85d51d37bc304498873dd204279742da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69b377dded7945329549bf5a64bf87b1" is_pre_election: true
I20260805 23:13:45.858554 10163 raft_consensus.cc:2781] T 748bd38edc9844d3941a5797032ce70d P 85d51d37bc304498873dd204279742da [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260805 23:13:45.858682 10163 raft_consensus.cc:482] T 748bd38edc9844d3941a5797032ce70d P 85d51d37bc304498873dd204279742da [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260805 23:13:45.858752 10163 raft_consensus.cc:3037] T 748bd38edc9844d3941a5797032ce70d P 85d51d37bc304498873dd204279742da [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:13:45.858863 10250 raft_consensus.cc:2454] T 748bd38edc9844d3941a5797032ce70d P 69b377dded7945329549bf5a64bf87b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85d51d37bc304498873dd204279742da in term 0.
I20260805 23:13:45.860409 10163 raft_consensus.cc:504] T 748bd38edc9844d3941a5797032ce70d P 85d51d37bc304498873dd204279742da [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd89afa0150a4437b4f2ea95593ccaf0" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 41629 } } peers { permanent_uuid: "69b377dded7945329549bf5a64bf87b1" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 44511 } } peers { permanent_uuid: "85d51d37bc304498873dd204279742da" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 45305 } }
I20260805 23:13:45.860976 10163 leader_election.cc:288] T 748bd38edc9844d3941a5797032ce70d P 85d51d37bc304498873dd204279742da [CANDIDATE]: Term 1 election: Requested vote from peers fd89afa0150a4437b4f2ea95593ccaf0 (127.25.92.195:41629), 69b377dded7945329549bf5a64bf87b1 (127.25.92.194:44511)
I20260805 23:13:45.861399 10250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "748bd38edc9844d3941a5797032ce70d" candidate_uuid: "85d51d37bc304498873dd204279742da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69b377dded7945329549bf5a64bf87b1"
I20260805 23:13:45.861584 10250 raft_consensus.cc:3037] T 748bd38edc9844d3941a5797032ce70d P 69b377dded7945329549bf5a64bf87b1 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:13:45.863336 10250 raft_consensus.cc:2454] T 748bd38edc9844d3941a5797032ce70d P 69b377dded7945329549bf5a64bf87b1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85d51d37bc304498873dd204279742da in term 1.
I20260805 23:13:45.863780 10043 leader_election.cc:302] T 748bd38edc9844d3941a5797032ce70d P 85d51d37bc304498873dd204279742da [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 69b377dded7945329549bf5a64bf87b1, 85d51d37bc304498873dd204279742da; no voters:
I20260805 23:13:45.863771 10385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "748bd38edc9844d3941a5797032ce70d" candidate_uuid: "85d51d37bc304498873dd204279742da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd89afa0150a4437b4f2ea95593ccaf0"
I20260805 23:13:45.863940 10385 raft_consensus.cc:3037] T 748bd38edc9844d3941a5797032ce70d P fd89afa0150a4437b4f2ea95593ccaf0 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:13:45.864037 10163 raft_consensus.cc:2781] T 748bd38edc9844d3941a5797032ce70d P 85d51d37bc304498873dd204279742da [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:13:45.864315 10163 raft_consensus.cc:686] T 748bd38edc9844d3941a5797032ce70d P 85d51d37bc304498873dd204279742da [term 1 LEADER]: Becoming Leader. State: Replica: 85d51d37bc304498873dd204279742da, State: Running, Role: LEADER
I20260805 23:13:45.864696 10163 consensus_queue.cc:237] T 748bd38edc9844d3941a5797032ce70d P 85d51d37bc304498873dd204279742da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd89afa0150a4437b4f2ea95593ccaf0" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 41629 } } peers { permanent_uuid: "69b377dded7945329549bf5a64bf87b1" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 44511 } } peers { permanent_uuid: "85d51d37bc304498873dd204279742da" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 45305 } }
I20260805 23:13:45.865734 10385 raft_consensus.cc:2454] T 748bd38edc9844d3941a5797032ce70d P fd89afa0150a4437b4f2ea95593ccaf0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85d51d37bc304498873dd204279742da in term 1.
I20260805 23:13:45.868234 9962 catalog_manager.cc:5495] T 748bd38edc9844d3941a5797032ce70d P 85d51d37bc304498873dd204279742da reported cstate change: term changed from 0 to 1, leader changed from <none> to 85d51d37bc304498873dd204279742da (127.25.92.193). New cstate: current_term: 1 leader_uuid: "85d51d37bc304498873dd204279742da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd89afa0150a4437b4f2ea95593ccaf0" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 41629 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "69b377dded7945329549bf5a64bf87b1" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 44511 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85d51d37bc304498873dd204279742da" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 45305 } health_report { overall_health: HEALTHY } } }
W20260805 23:13:45.885068 10299 tablet.cc:2369] T 748bd38edc9844d3941a5797032ce70d P 69b377dded7945329549bf5a64bf87b1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260805 23:13:45.933588 10459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:45.929292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47380 (local address 127.25.92.193:45305)
0805 23:13:45.929659 (+ 367us) server_negotiation.cc:206] Beginning negotiation
0805 23:13:45.929667 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0805 23:13:45.929680 (+ 13us) server_negotiation.cc:407] Connection header received
0805 23:13:45.929760 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:13:45.929762 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:13:45.929819 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:13:45.929960 (+ 141us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 23:13:45.930246 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:13:45.930837 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:13:45.931801 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:13:45.932150 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:13:45.932284 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:13:45.932591 (+ 307us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 23:13:45.933061 (+ 470us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 23:13:45.933200 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0805 23:13:45.933270 (+ 70us) server_negotiation.cc:299] Negotiation successful
0805 23:13:45.933329 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":120,"threads_started":1}
I20260805 23:13:45.937992 10250 raft_consensus.cc:1261] T 748bd38edc9844d3941a5797032ce70d P 69b377dded7945329549bf5a64bf87b1 [term 1 FOLLOWER]: Refusing update from remote peer 85d51d37bc304498873dd204279742da: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:13:45.938006 10385 raft_consensus.cc:1261] T 748bd38edc9844d3941a5797032ce70d P fd89afa0150a4437b4f2ea95593ccaf0 [term 1 FOLLOWER]: Refusing update from remote peer 85d51d37bc304498873dd204279742da: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:13:45.938776 10455 consensus_queue.cc:1035] T 748bd38edc9844d3941a5797032ce70d P 85d51d37bc304498873dd204279742da [LEADER]: Connected to new peer: Peer: permanent_uuid: "69b377dded7945329549bf5a64bf87b1" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 44511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:13:45.938968 10163 consensus_queue.cc:1035] T 748bd38edc9844d3941a5797032ce70d P 85d51d37bc304498873dd204279742da [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd89afa0150a4437b4f2ea95593ccaf0" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 41629 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:13:45.949295 10462 mvcc.cc:205] Tried to move back new op lower bound from 7315339779834060800 to 7315339779545587712. Current Snapshot: MvccSnapshot[applied={T|T < 7315339779834060800}]
I20260805 23:13:45.952668 10461 mvcc.cc:205] Tried to move back new op lower bound from 7315339779834060800 to 7315339779545587712. Current Snapshot: MvccSnapshot[applied={T|T < 7315339779834060800}]
I20260805 23:13:45.953068 10460 mvcc.cc:205] Tried to move back new op lower bound from 7315339779834060800 to 7315339779545587712. Current Snapshot: MvccSnapshot[applied={T|T < 7315339779834060800 or (T in {7315339779834060800})}]
I20260805 23:13:45.959410 9962 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38576:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:13:45.959599 9962 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38576:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:13:45.962061 9962 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 242b9070a63b4c049d7c7563d64f021b: Sending DeleteTablet for 3 replicas of tablet 748bd38edc9844d3941a5797032ce70d
I20260805 23:13:45.962852 10365 tablet_service.cc:1514] Processing DeleteTablet for tablet 748bd38edc9844d3941a5797032ce70d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:13:45 UTC) from {username='kudu', principal='kudu/127.25.92.254@KRBTEST.COM'} at 127.0.0.1:41700
I20260805 23:13:45.962846 10230 tablet_service.cc:1514] Processing DeleteTablet for tablet 748bd38edc9844d3941a5797032ce70d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:13:45 UTC) from {username='kudu', principal='kudu/127.25.92.254@KRBTEST.COM'} at 127.0.0.1:41238
I20260805 23:13:45.963717 10471 tablet_replica.cc:330] T 748bd38edc9844d3941a5797032ce70d P fd89afa0150a4437b4f2ea95593ccaf0: stopping tablet replica
I20260805 23:13:45.963783 10089 tablet_service.cc:1514] Processing DeleteTablet for tablet 748bd38edc9844d3941a5797032ce70d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:13:45 UTC) from {username='kudu', principal='kudu/127.25.92.254@KRBTEST.COM'} at 127.0.0.1:47372
I20260805 23:13:45.964107 10471 raft_consensus.cc:2229] T 748bd38edc9844d3941a5797032ce70d P fd89afa0150a4437b4f2ea95593ccaf0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 23:13:45.964468 10471 raft_consensus.cc:2258] T 748bd38edc9844d3941a5797032ce70d P fd89afa0150a4437b4f2ea95593ccaf0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:13:45.964572 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 10002
I20260805 23:13:45.966722 10471 ts_tablet_manager.cc:1905] T 748bd38edc9844d3941a5797032ce70d P fd89afa0150a4437b4f2ea95593ccaf0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:13:45.969755 10470 tablet_replica.cc:330] T 748bd38edc9844d3941a5797032ce70d P 69b377dded7945329549bf5a64bf87b1: stopping tablet replica
I20260805 23:13:45.970147 10470 raft_consensus.cc:2229] T 748bd38edc9844d3941a5797032ce70d P 69b377dded7945329549bf5a64bf87b1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 23:13:45.970614 10470 raft_consensus.cc:2258] T 748bd38edc9844d3941a5797032ce70d P 69b377dded7945329549bf5a64bf87b1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:13:45.972091 10471 ts_tablet_manager.cc:1918] T 748bd38edc9844d3941a5797032ce70d P fd89afa0150a4437b4f2ea95593ccaf0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 23:13:45.972209 10471 log.cc:1196] T 748bd38edc9844d3941a5797032ce70d P fd89afa0150a4437b4f2ea95593ccaf0: Deleting WAL directory at /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-2/wal/wals/748bd38edc9844d3941a5797032ce70d
I20260805 23:13:45.973253 10471 ts_tablet_manager.cc:1939] T 748bd38edc9844d3941a5797032ce70d P fd89afa0150a4437b4f2ea95593ccaf0: Deleting consensus metadata
I20260805 23:13:45.974067 10470 ts_tablet_manager.cc:1905] T 748bd38edc9844d3941a5797032ce70d P 69b377dded7945329549bf5a64bf87b1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:13:45.974929 9948 catalog_manager.cc:4841] TS fd89afa0150a4437b4f2ea95593ccaf0 (127.25.92.195:41629): tablet 748bd38edc9844d3941a5797032ce70d (table test-table [id=e2997c9f521a4d7f833d49c342d63546]) successfully deleted
W20260805 23:13:45.976369 9947 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.25.92.193:45305 (error 108)
I20260805 23:13:45.977748 10444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:45.977444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.193:45305 (local address 127.0.0.1:47394)
0805 23:13:45.977602 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0805 23:13:45.977677 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.193:45305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260805 23:13:45.977779 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 10161
W20260805 23:13:45.978760 9947 catalog_manager.cc:4584] TS 85d51d37bc304498873dd204279742da (127.25.92.193:45305): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 748bd38edc9844d3941a5797032ce70d: Network error: Client connection negotiation failed: client connection to 127.25.92.193:45305: connect: Connection refused (error 111)
I20260805 23:13:45.987402 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 10301
I20260805 23:13:45.990715 10444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:13:45.990398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.194:44511 (local address 127.0.0.1:41250)
0805 23:13:45.990551 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0805 23:13:45.990627 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.194:44511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260805 23:13:45.994932 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 9929
I20260805 23:14:06.002321 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.254:34379
--webserver_interface=127.25.92.254
--webserver_port=40893
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab
--principal=kudu/127.25.92.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:33407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.92.254:34379
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:14:06.115911 10475 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:14:06.116204 10475 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:14:06.116245 10475 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:14:06.120419 10475 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260805 23:14:06.120487 10475 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260805 23:14:06.120503 10475 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260805 23:14:06.120517 10475 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:14:06.120537 10475 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:14:06.120555 10475 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:14:06.120630 10475 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:14:06.120661 10475 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:14:06.126201 10475 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:33407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.25.92.254:34379
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab
--principal=kudu/127.25.92.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.254:34379
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.92.254
--webserver_port=40893
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10475
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:14:06.127403 10475 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:14:06.128820 10475 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:14:06.137805 10483 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:14:06.137820 10481 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:14:06.137820 10480 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:14:06.137846 10475 server_base.cc:1034] running on GCE node
I20260805 23:14:06.138810 10475 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:14:06.139991 10475 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:14:06.141192 10475 hybrid_clock.cc:648] HybridClock initialized: now 1785971646141167 us; error 52 us; skew 500 ppm
Aug 05 23:14:06 dist-test-slave-ss4n krb5kdc[9361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971646, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:14:06.150753 10475 init.cc:377] Logged in from keytab as kudu/127.25.92.254@KRBTEST.COM (short username kudu)
I20260805 23:14:06.151813 10475 webserver.cc:460] Webserver started at http://127.25.92.254:40893/ using document root <none> and password file <none>
I20260805 23:14:06.152362 10475 fs_manager.cc:359] Metadata directory not provided
I20260805 23:14:06.152427 10475 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:14:06.156472 10475 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260805 23:14:06.158771 10492 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:14:06.160110 10475 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260805 23:14:06.160264 10475 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/master-0/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/master-0/wal
uuid: "242b9070a63b4c049d7c7563d64f021b"
format_stamp: "Formatted at 2026-08-05 23:13:23 on dist-test-slave-ss4n"
server_key: "9cde2619d4310f38000f1ef33c06a78a"
server_key_iv: "f6299010fe621bb41c17e89177c7cdaa"
server_key_version: "encryptionkey@0"
I20260805 23:14:06.160919 10475 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:14:06.181734 10475 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:14:06.185352 10475 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:14:06.185618 10475 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:14:06.193280 10475 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.254:34379
I20260805 23:14:06.193708 10546 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.254:34379 every 8 connection(s)
I20260805 23:14:06.194468 10475 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
I20260805 23:14:06.199059 10547 sys_catalog.cc:263] Verifying existing consensus state
I20260805 23:14:06.200037 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 10475
I20260805 23:14:06.200531 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.92.193:45305
--local_ip_for_outbound_sockets=127.25.92.193
--tserver_master_addrs=127.25.92.254:34379
--webserver_port=43515
--webserver_interface=127.25.92.193
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.193.keytab
--principal=kudu/127.25.92.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:33407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260805 23:14:06.201040 10547 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 242b9070a63b4c049d7c7563d64f021b: Bootstrap starting.
I20260805 23:14:06.219681 10547 log.cc:824] T 00000000000000000000000000000000 P 242b9070a63b4c049d7c7563d64f021b: Log is configured to *not* fsync() on all Append() calls
I20260805 23:14:06.231580 10547 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 242b9070a63b4c049d7c7563d64f021b: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260805 23:14:06.232065 10547 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 242b9070a63b4c049d7c7563d64f021b: Bootstrap complete.
I20260805 23:14:06.236512 10547 raft_consensus.cc:348] T 00000000000000000000000000000000 P 242b9070a63b4c049d7c7563d64f021b [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "242b9070a63b4c049d7c7563d64f021b" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 34379 } }
I20260805 23:14:06.236960 10547 raft_consensus.cc:729] T 00000000000000000000000000000000 P 242b9070a63b4c049d7c7563d64f021b [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242b9070a63b4c049d7c7563d64f021b, State: Initialized, Role: FOLLOWER
I20260805 23:14:06.237533 10547 consensus_queue.cc:260] T 00000000000000000000000000000000 P 242b9070a63b4c049d7c7563d64f021b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "242b9070a63b4c049d7c7563d64f021b" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 34379 } }
I20260805 23:14:06.237736 10547 raft_consensus.cc:388] T 00000000000000000000000000000000 P 242b9070a63b4c049d7c7563d64f021b [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:14:06.237833 10547 raft_consensus.cc:482] T 00000000000000000000000000000000 P 242b9070a63b4c049d7c7563d64f021b [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:14:06.237937 10547 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 242b9070a63b4c049d7c7563d64f021b [term 2 FOLLOWER]: Advancing to term 3
I20260805 23:14:06.240312 10547 raft_consensus.cc:504] T 00000000000000000000000000000000 P 242b9070a63b4c049d7c7563d64f021b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "242b9070a63b4c049d7c7563d64f021b" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 34379 } }
I20260805 23:14:06.240782 10547 leader_election.cc:302] T 00000000000000000000000000000000 P 242b9070a63b4c049d7c7563d64f021b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 242b9070a63b4c049d7c7563d64f021b; no voters:
I20260805 23:14:06.241183 10547 leader_election.cc:288] T 00000000000000000000000000000000 P 242b9070a63b4c049d7c7563d64f021b [CANDIDATE]: Term 3 election: Requested vote from peers
I20260805 23:14:06.241276 10553 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 242b9070a63b4c049d7c7563d64f021b [term 3 FOLLOWER]: Leader election won for term 3
I20260805 23:14:06.241513 10553 raft_consensus.cc:686] T 00000000000000000000000000000000 P 242b9070a63b4c049d7c7563d64f021b [term 3 LEADER]: Becoming Leader. State: Replica: 242b9070a63b4c049d7c7563d64f021b, State: Running, Role: LEADER
I20260805 23:14:06.241892 10553 consensus_queue.cc:237] T 00000000000000000000000000000000 P 242b9070a63b4c049d7c7563d64f021b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "242b9070a63b4c049d7c7563d64f021b" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 34379 } }
I20260805 23:14:06.242532 10547 sys_catalog.cc:564] T 00000000000000000000000000000000 P 242b9070a63b4c049d7c7563d64f021b [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:14:06.243456 10555 sys_catalog.cc:455] T 00000000000000000000000000000000 P 242b9070a63b4c049d7c7563d64f021b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 242b9070a63b4c049d7c7563d64f021b. Latest consensus state: current_term: 3 leader_uuid: "242b9070a63b4c049d7c7563d64f021b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "242b9070a63b4c049d7c7563d64f021b" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 34379 } } }
I20260805 23:14:06.243593 10555 sys_catalog.cc:458] T 00000000000000000000000000000000 P 242b9070a63b4c049d7c7563d64f021b [sys.catalog]: This master's current role is: LEADER
I20260805 23:14:06.243858 10554 sys_catalog.cc:455] T 00000000000000000000000000000000 P 242b9070a63b4c049d7c7563d64f021b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "242b9070a63b4c049d7c7563d64f021b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "242b9070a63b4c049d7c7563d64f021b" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 34379 } } }
I20260805 23:14:06.243929 10554 sys_catalog.cc:458] T 00000000000000000000000000000000 P 242b9070a63b4c049d7c7563d64f021b [sys.catalog]: This master's current role is: LEADER
I20260805 23:14:06.243952 10562 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:14:06.249809 10562 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:14:06.250687 10562 catalog_manager.cc:1259] Loaded cluster ID: bd8331ed9dd64faca326013e7f711783
I20260805 23:14:06.250785 10562 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:14:06.252767 10562 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:14:06.253443 10562 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 242b9070a63b4c049d7c7563d64f021b: Loaded TSK: 0
I20260805 23:14:06.253577 10562 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 242b9070a63b4c049d7c7563d64f021b: Loaded TSK: 1
F20260805 23:14:06.254241 10562 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f5d6e6ad9ad google::LogMessage::Fail() at ??:0
@ 0x7f5d6e6b088a google::LogMessage::SendToLog() at ??:0
@ 0x7f5d6e6ad4de google::LogMessage::Flush() at ??:0
@ 0x7f5d6e6aea19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f5d7137118d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f5d71369f0d kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f5d790127bb kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f5d78fec99c kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f5d78fed178 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlvE3_clEv at ??:0
@ 0x7f5d7902edf8 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f5d7904d82b std::function<>::operator()() at ??:0
@ 0x7f5d78fecb4f _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f5d78fedf3e kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f5d78fea579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f5d7902e640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f5d708ff3d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f5d708ffc83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f5d709013f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f5d708f0512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f5d71f346db start_thread at ??:0
@ 0x7f5d6d1ab61f clone at ??:0
W20260805 23:14:06.352669 10551 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:14:06.353120 10551 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:14:06.353204 10551 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:14:06.361282 10551 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:14:06.361380 10551 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:14:06.361524 10551 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.92.193
I20260805 23:14:06.371812 10551 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:33407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.193.keytab
--principal=kudu/127.25.92.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.92.193:45305
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.92.193
--webserver_port=43515
--tserver_master_addrs=127.25.92.254:34379
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10551
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.92.193
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:14:06.373453 10551 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:14:06.374989 10551 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:14:06.389720 10600 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:14:06.389719 10602 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:14:06.389778 10599 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:14:06.390625 10551 server_base.cc:1034] running on GCE node
I20260805 23:14:06.391111 10551 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:14:06.391886 10551 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:14:06.393137 10551 hybrid_clock.cc:648] HybridClock initialized: now 1785971646393095 us; error 53 us; skew 500 ppm
Aug 05 23:14:06 dist-test-slave-ss4n krb5kdc[9361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971646, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:14:06.396385 10551 init.cc:377] Logged in from keytab as kudu/127.25.92.193@KRBTEST.COM (short username kudu)
I20260805 23:14:06.397528 10551 webserver.cc:460] Webserver started at http://127.25.92.193:43515/ using document root <none> and password file <none>
I20260805 23:14:06.398232 10551 fs_manager.cc:359] Metadata directory not provided
I20260805 23:14:06.398305 10551 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:14:06.402783 10551 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260805 23:14:06.405287 10609 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:14:06.406745 10551 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260805 23:14:06.406919 10551 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-0/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-0/wal
uuid: "85d51d37bc304498873dd204279742da"
format_stamp: "Formatted at 2026-08-05 23:13:24 on dist-test-slave-ss4n"
server_key: "b2eecfd72b6f80b9d7e2d72f01a973d8"
server_key_iv: "f4d7aa860e2f69542f8b62006293d9a9"
server_key_version: "encryptionkey@0"
I20260805 23:14:06.407546 10551 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:14:06.430387 10551 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:14:06.434120 10551 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:14:06.434360 10551 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:14:06.435050 10551 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:14:06.436298 10616 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260805 23:14:06.441579 10551 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260805 23:14:06.441663 10551 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260805 23:14:06.441762 10551 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260805 23:14:06.443682 10551 ts_tablet_manager.cc:610] Registered 1 tablets
I20260805 23:14:06.443756 10551 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260805 23:14:06.444010 10616 tablet_bootstrap.cc:492] T 748bd38edc9844d3941a5797032ce70d P 85d51d37bc304498873dd204279742da: Bootstrap starting.
I20260805 23:14:06.455668 10551 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.193:45305
I20260805 23:14:06.456144 10729 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.193:45305 every 8 connection(s)
I20260805 23:14:06.457207 10551 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-0/data/info.pb
I20260805 23:14:06.459344 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 10551
I20260805 23:14:06.459898 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.92.194:44511
--local_ip_for_outbound_sockets=127.25.92.194
--tserver_master_addrs=127.25.92.254:34379
--webserver_port=44771
--webserver_interface=127.25.92.194
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.194.keytab
--principal=kudu/127.25.92.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:33407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260805 23:14:06.467669 10616 log.cc:824] T 748bd38edc9844d3941a5797032ce70d P 85d51d37bc304498873dd204279742da: Log is configured to *not* fsync() on all Append() calls
I20260805 23:14:06.478442 10616 tablet_bootstrap.cc:492] T 748bd38edc9844d3941a5797032ce70d P 85d51d37bc304498873dd204279742da: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260805 23:14:06.478925 10616 tablet_bootstrap.cc:492] T 748bd38edc9844d3941a5797032ce70d P 85d51d37bc304498873dd204279742da: Bootstrap complete.
I20260805 23:14:06.480394 10616 ts_tablet_manager.cc:1397] T 748bd38edc9844d3941a5797032ce70d P 85d51d37bc304498873dd204279742da: Time spent bootstrapping tablet: real 0.037s user 0.025s sys 0.008s
I20260805 23:14:06.484009 10616 raft_consensus.cc:348] T 748bd38edc9844d3941a5797032ce70d P 85d51d37bc304498873dd204279742da [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd89afa0150a4437b4f2ea95593ccaf0" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 41629 } } peers { permanent_uuid: "69b377dded7945329549bf5a64bf87b1" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 44511 } } peers { permanent_uuid: "85d51d37bc304498873dd204279742da" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 45305 } }
I20260805 23:14:06.484601 10616 raft_consensus.cc:729] T 748bd38edc9844d3941a5797032ce70d P 85d51d37bc304498873dd204279742da [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85d51d37bc304498873dd204279742da, State: Initialized, Role: FOLLOWER
I20260805 23:14:06.485147 10616 consensus_queue.cc:260] T 748bd38edc9844d3941a5797032ce70d P 85d51d37bc304498873dd204279742da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd89afa0150a4437b4f2ea95593ccaf0" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 41629 } } peers { permanent_uuid: "69b377dded7945329549bf5a64bf87b1" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 44511 } } peers { permanent_uuid: "85d51d37bc304498873dd204279742da" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 45305 } }
I20260805 23:14:06.486335 10616 ts_tablet_manager.cc:1428] T 748bd38edc9844d3941a5797032ce70d P 85d51d37bc304498873dd204279742da: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
W20260805 23:14:06.524542 10732 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:14:06.459881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:34379 (local address 127.25.92.193:40407)
0805 23:14:06.461298 (+ 1417us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:06.461372 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0805 23:14:06.462697 (+ 1325us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:14:06.523453 (+ 60756us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:34379: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":1174,"thread_start_us":168,"threads_started":1}
W20260805 23:14:06.525252 10730 heartbeater.cc:644] Failed to heartbeat to 127.25.92.254:34379 (0 consecutive failures): Network error: Failed to ping master at 127.25.92.254:34379: Client connection negotiation failed: client connection to 127.25.92.254:34379: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260805 23:14:06.526130 10732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:06.525802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:34379 (local address 127.25.92.193:45145)
0805 23:14:06.525971 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:06.526057 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:34379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260805 23:14:06.526903 10732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:06.526631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:34379 (local address 127.25.92.193:49815)
0805 23:14:06.526781 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:06.526839 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:34379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260805 23:14:06.527125 10730 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260805 23:14:06.589730 10733 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:14:06.590071 10733 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:14:06.590148 10733 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:14:06.594275 10733 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:14:06.594383 10733 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:14:06.594497 10733 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.92.194
I20260805 23:14:06.600181 10733 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:33407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.194.keytab
--principal=kudu/127.25.92.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.92.194:44511
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.92.194
--webserver_port=44771
--tserver_master_addrs=127.25.92.254:34379
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10733
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.92.194
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:14:06.601348 10733 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:14:06.602502 10733 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:14:06.612524 10742 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:14:06.612576 10740 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:14:06.612576 10745 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:14:06.612913 10733 server_base.cc:1034] running on GCE node
I20260805 23:14:06.613369 10733 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:14:06.614089 10733 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:14:06.615330 10733 hybrid_clock.cc:648] HybridClock initialized: now 1785971646615310 us; error 34 us; skew 500 ppm
Aug 05 23:14:06 dist-test-slave-ss4n krb5kdc[9361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971646, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:14:06.618400 10733 init.cc:377] Logged in from keytab as kudu/127.25.92.194@KRBTEST.COM (short username kudu)
I20260805 23:14:06.619305 10733 webserver.cc:460] Webserver started at http://127.25.92.194:44771/ using document root <none> and password file <none>
I20260805 23:14:06.619863 10733 fs_manager.cc:359] Metadata directory not provided
I20260805 23:14:06.619923 10733 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:14:06.624089 10733 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260805 23:14:06.626672 10752 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:14:06.628033 10733 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260805 23:14:06.628149 10733 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-1/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-1/wal
uuid: "69b377dded7945329549bf5a64bf87b1"
format_stamp: "Formatted at 2026-08-05 23:13:24 on dist-test-slave-ss4n"
server_key: "a6fb1dbdbb49b2b0b0e8f2fdd7d0fd6d"
server_key_iv: "38dbd67ee19f383f3a91fe3b8aafd53e"
server_key_version: "encryptionkey@0"
I20260805 23:14:06.628758 10733 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:14:06.658229 10733 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:14:06.661720 10733 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:14:06.661890 10733 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:14:06.662473 10733 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:14:06.663615 10759 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
W20260805 23:14:06.667757 10759 ts_tablet_manager.cc:1852] T 748bd38edc9844d3941a5797032ce70d P 69b377dded7945329549bf5a64bf87b1: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260805 23:14:06.668193 10759 ts_tablet_manager.cc:1905] T 748bd38edc9844d3941a5797032ce70d P 69b377dded7945329549bf5a64bf87b1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:14:06.672537 10759 ts_tablet_manager.cc:1918] T 748bd38edc9844d3941a5797032ce70d P 69b377dded7945329549bf5a64bf87b1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 23:14:06.672652 10759 log.cc:1196] T 748bd38edc9844d3941a5797032ce70d P 69b377dded7945329549bf5a64bf87b1: Deleting WAL directory at /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-1/wal/wals/748bd38edc9844d3941a5797032ce70d
I20260805 23:14:06.673069 10759 ts_tablet_manager.cc:1939] T 748bd38edc9844d3941a5797032ce70d P 69b377dded7945329549bf5a64bf87b1: Deleting consensus metadata
I20260805 23:14:06.673449 10733 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260805 23:14:06.673588 10733 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.010s user 0.001s sys 0.000s
I20260805 23:14:06.673707 10733 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:14:06.673756 10733 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:14:06.683632 10733 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.194:44511
I20260805 23:14:06.683661 10871 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.194:44511 every 8 connection(s)
I20260805 23:14:06.684592 10733 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-1/data/info.pb
I20260805 23:14:06.688259 10874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:06.686884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:34379 (local address 127.25.92.194:57827)
0805 23:14:06.687404 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:06.687518 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:34379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":124,"threads_started":1}
W20260805 23:14:06.688736 10872 heartbeater.cc:644] Failed to heartbeat to 127.25.92.254:34379 (0 consecutive failures): Network error: Failed to ping master at 127.25.92.254:34379: Client connection negotiation failed: client connection to 127.25.92.254:34379: connect: Connection refused (error 111)
I20260805 23:14:06.689430 10874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:06.689192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:34379 (local address 127.25.92.194:46287)
0805 23:14:06.689314 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:06.689368 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:34379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260805 23:14:06.690148 10874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:06.689925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:34379 (local address 127.25.92.194:58365)
0805 23:14:06.690028 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:06.690093 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:34379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260805 23:14:06.690343 10872 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260805 23:14:06.691738 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 10733
I20260805 23:14:06.692162 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.92.195:41629
--local_ip_for_outbound_sockets=127.25.92.195
--tserver_master_addrs=127.25.92.254:34379
--webserver_port=39537
--webserver_interface=127.25.92.195
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.195.keytab
--principal=kudu/127.25.92.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:33407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:14:06.811030 10875 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:14:06.811364 10875 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:14:06.811416 10875 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:14:06.815735 10875 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:14:06.815821 10875 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:14:06.815886 10875 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.92.195
I20260805 23:14:06.821659 10875 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:33407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.195.keytab
--principal=kudu/127.25.92.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.92.195:41629
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.92.195
--webserver_port=39537
--tserver_master_addrs=127.25.92.254:34379
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10875
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.92.195
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:14:06.822892 10875 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:14:06.823988 10875 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:14:06.834657 10881 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:14:06.834712 10882 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:14:06.834548 10884 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:14:06.834789 10875 server_base.cc:1034] running on GCE node
I20260805 23:14:06.835356 10875 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:14:06.836081 10875 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:14:06.837311 10875 hybrid_clock.cc:648] HybridClock initialized: now 1785971646837296 us; error 29 us; skew 500 ppm
Aug 05 23:14:06 dist-test-slave-ss4n krb5kdc[9361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971646, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:14:06.840552 10875 init.cc:377] Logged in from keytab as kudu/127.25.92.195@KRBTEST.COM (short username kudu)
I20260805 23:14:06.841692 10875 webserver.cc:460] Webserver started at http://127.25.92.195:39537/ using document root <none> and password file <none>
I20260805 23:14:06.842366 10875 fs_manager.cc:359] Metadata directory not provided
I20260805 23:14:06.842491 10875 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:14:06.847386 10875 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.003s
I20260805 23:14:06.849725 10892 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:14:06.851104 10875 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260805 23:14:06.851287 10875 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-2/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-2/wal
uuid: "fd89afa0150a4437b4f2ea95593ccaf0"
format_stamp: "Formatted at 2026-08-05 23:13:24 on dist-test-slave-ss4n"
server_key: "dad15bf071517ab654fa7c853f5edbfb"
server_key_iv: "3cf942aaac65f2be89eed7fbce6f0497"
server_key_version: "encryptionkey@0"
I20260805 23:14:06.851904 10875 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:14:06.863968 10875 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:14:06.867905 10875 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:14:06.868142 10875 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:14:06.868786 10875 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:14:06.869735 10875 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:14:06.869807 10875 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:14:06.869887 10875 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:14:06.869938 10875 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:14:06.879307 10875 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.195:41629
I20260805 23:14:06.879333 11008 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.195:41629 every 8 connection(s)
I20260805 23:14:06.880344 10875 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971406251336-25971-0/minicluster-data/ts-2/data/info.pb
I20260805 23:14:06.884452 11011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:06.882739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:34379 (local address 127.25.92.195:48937)
0805 23:14:06.883473 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:06.883590 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:34379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":534,"thread_start_us":147,"threads_started":1}
W20260805 23:14:06.885067 11009 heartbeater.cc:644] Failed to heartbeat to 127.25.92.254:34379 (0 consecutive failures): Network error: Failed to ping master at 127.25.92.254:34379: Client connection negotiation failed: client connection to 127.25.92.254:34379: connect: Connection refused (error 111)
I20260805 23:14:06.886050 11011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:06.885809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:34379 (local address 127.25.92.195:33691)
0805 23:14:06.885937 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:06.885993 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:34379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260805 23:14:06.886798 11011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:06.886582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:34379 (local address 127.25.92.195:54875)
0805 23:14:06.886702 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:06.886748 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:34379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260805 23:14:06.887024 11009 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260805 23:14:06.889971 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 10875
I20260805 23:14:07.528239 11015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:07.527778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:34379 (local address 127.25.92.193:48315)
0805 23:14:07.528060 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:07.528118 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:34379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":83,"threads_started":1}
I20260805 23:14:07.691583 11016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:07.690926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:34379 (local address 127.25.92.194:47321)
0805 23:14:07.691379 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:07.691453 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:34379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":79,"threads_started":1}
I20260805 23:14:07.793151 11017 raft_consensus.cc:482] T 748bd38edc9844d3941a5797032ce70d P 85d51d37bc304498873dd204279742da [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 23:14:07.793383 11017 raft_consensus.cc:504] T 748bd38edc9844d3941a5797032ce70d P 85d51d37bc304498873dd204279742da [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd89afa0150a4437b4f2ea95593ccaf0" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 41629 } } peers { permanent_uuid: "69b377dded7945329549bf5a64bf87b1" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 44511 } } peers { permanent_uuid: "85d51d37bc304498873dd204279742da" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 45305 } }
I20260805 23:14:07.794584 11017 leader_election.cc:288] T 748bd38edc9844d3941a5797032ce70d P 85d51d37bc304498873dd204279742da [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fd89afa0150a4437b4f2ea95593ccaf0 (127.25.92.195:41629), 69b377dded7945329549bf5a64bf87b1 (127.25.92.194:44511)
Aug 05 23:14:07 dist-test-slave-ss4n krb5kdc[9361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971646, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.193@KRBTEST.COM for kudu/127.25.92.195@KRBTEST.COM
Aug 05 23:14:07 dist-test-slave-ss4n krb5kdc[9361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971646, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.193@KRBTEST.COM for kudu/127.25.92.194@KRBTEST.COM
I20260805 23:14:07.812052 11018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:07.794804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.195:41629 (local address 127.25.92.193:51311)
0805 23:14:07.795214 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:07.795233 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0805 23:14:07.795357 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:14:07.796878 (+ 1521us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:14:07.796885 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:14:07.797454 (+ 569us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:14:07.797974 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:14:07.797989 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:14:07.799305 (+ 1316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:14:07.799312 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:14:07.800240 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:14:07.800250 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:14:07.800974 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:14:07.800978 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:14:07.801187 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:07.802065 (+ 878us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:14:07.802089 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:14:07.804426 (+ 2337us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:14:07.808398 (+ 3972us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:14:07.808407 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:14:07.808425 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:14:07.808844 (+ 419us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:14:07.809617 (+ 773us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:14:07.809621 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:14:07.809624 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:14:07.809852 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:14:07.810809 (+ 957us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:14:07.810816 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:14:07.811281 (+ 465us) client_negotiation.cc:769] Sending connection context
0805 23:14:07.811495 (+ 214us) client_negotiation.cc:240] Negotiation successful
0805 23:14:07.811831 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"mutex_wait_us":104,"spinlock_wait_cycles":2432}
I20260805 23:14:07.812083 11019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:07.794968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.193:51311 (local address 127.25.92.195:41629)
0805 23:14:07.795428 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0805 23:14:07.795442 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0805 23:14:07.795478 (+ 36us) server_negotiation.cc:407] Connection header received
0805 23:14:07.796161 (+ 683us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:14:07.796167 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:14:07.796502 (+ 335us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:14:07.796745 (+ 243us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:14:07.798182 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:07.799152 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:07.800399 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:07.800837 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:07.801004 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:07.804732 (+ 3728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:14:07.804741 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:14:07.804761 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:14:07.804807 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:14:07.808195 (+ 3388us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:14:07.808983 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:14:07.808994 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:14:07.809002 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:14:07.809203 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:14:07.809957 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:14:07.809966 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:14:07.809971 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:14:07.810671 (+ 700us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:14:07.810825 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0805 23:14:07.811648 (+ 823us) server_negotiation.cc:299] Negotiation successful
0805 23:14:07.811924 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":128,"threads_started":1}
W20260805 23:14:07.815790 10611 leader_election.cc:341] T 748bd38edc9844d3941a5797032ce70d P 85d51d37bc304498873dd204279742da [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fd89afa0150a4437b4f2ea95593ccaf0 (127.25.92.195:41629): Not found: Tablet not found: 748bd38edc9844d3941a5797032ce70d
I20260805 23:14:07.819878 11015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:07.794927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.194:44511 (local address 127.25.92.193:43641)
0805 23:14:07.795362 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:07.795377 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0805 23:14:07.795480 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:14:07.796883 (+ 1403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:14:07.796886 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:14:07.797448 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:14:07.797974 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:14:07.797989 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:14:07.799368 (+ 1379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:14:07.799371 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:14:07.800235 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:14:07.800244 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:14:07.801015 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:14:07.801018 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:14:07.801191 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:07.802156 (+ 965us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:14:07.802190 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:14:07.810998 (+ 8808us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:14:07.817444 (+ 6446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:14:07.817450 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:14:07.817453 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:14:07.817802 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:14:07.818262 (+ 460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:14:07.818265 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:14:07.818267 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:14:07.818363 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:14:07.819280 (+ 917us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:14:07.819283 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:14:07.819411 (+ 128us) client_negotiation.cc:769] Sending connection context
0805 23:14:07.819523 (+ 112us) client_negotiation.cc:240] Negotiation successful
0805 23:14:07.819691 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":354,"mutex_wait_us":92,"thread_start_us":96,"threads_started":1}
I20260805 23:14:07.820369 11020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:07.795098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.193:43641 (local address 127.25.92.194:44511)
0805 23:14:07.795390 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0805 23:14:07.795410 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0805 23:14:07.795462 (+ 52us) server_negotiation.cc:407] Connection header received
0805 23:14:07.796273 (+ 811us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:14:07.796279 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:14:07.796598 (+ 319us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:14:07.796920 (+ 322us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:14:07.798173 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:07.799200 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:07.800360 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:07.800850 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:07.801046 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:07.811375 (+ 10329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:14:07.811385 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:14:07.811408 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:14:07.811457 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:14:07.817269 (+ 5812us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:14:07.817970 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:14:07.817979 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:14:07.817985 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:14:07.818165 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:14:07.818481 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:14:07.818489 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:14:07.818492 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:14:07.819173 (+ 681us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:14:07.819313 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0805 23:14:07.819697 (+ 384us) server_negotiation.cc:299] Negotiation successful
0805 23:14:07.820161 (+ 464us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":56,"threads_started":1}
W20260805 23:14:07.823348 10613 leader_election.cc:341] T 748bd38edc9844d3941a5797032ce70d P 85d51d37bc304498873dd204279742da [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 69b377dded7945329549bf5a64bf87b1 (127.25.92.194:44511): Not found: Tablet not found: 748bd38edc9844d3941a5797032ce70d
I20260805 23:14:07.823482 10613 leader_election.cc:302] T 748bd38edc9844d3941a5797032ce70d P 85d51d37bc304498873dd204279742da [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85d51d37bc304498873dd204279742da; no voters: 69b377dded7945329549bf5a64bf87b1, fd89afa0150a4437b4f2ea95593ccaf0
I20260805 23:14:07.823843 11017 raft_consensus.cc:2726] T 748bd38edc9844d3941a5797032ce70d P 85d51d37bc304498873dd204279742da [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260805 23:14:07.888231 11021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:07.887666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:34379 (local address 127.25.92.195:46877)
0805 23:14:07.887981 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:07.888064 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:34379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":87,"threads_started":1}
I20260805 23:14:08.529793 11022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:08.529135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:34379 (local address 127.25.92.193:58303)
0805 23:14:08.529596 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:08.529665 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:34379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":96,"threads_started":1}
I20260805 23:14:08.693049 11023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:08.692597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:34379 (local address 127.25.92.194:37239)
0805 23:14:08.692871 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:08.692934 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:34379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":86,"threads_started":1}
I20260805 23:14:08.889870 11024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:08.889265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:34379 (local address 127.25.92.195:52421)
0805 23:14:08.889678 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:08.889747 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:34379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":79,"threads_started":1}
I20260805 23:14:09.465296 11025 raft_consensus.cc:482] T 748bd38edc9844d3941a5797032ce70d P 85d51d37bc304498873dd204279742da [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 23:14:09.465484 11025 raft_consensus.cc:504] T 748bd38edc9844d3941a5797032ce70d P 85d51d37bc304498873dd204279742da [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd89afa0150a4437b4f2ea95593ccaf0" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 41629 } } peers { permanent_uuid: "69b377dded7945329549bf5a64bf87b1" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 44511 } } peers { permanent_uuid: "85d51d37bc304498873dd204279742da" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 45305 } }
I20260805 23:14:09.466046 11025 leader_election.cc:288] T 748bd38edc9844d3941a5797032ce70d P 85d51d37bc304498873dd204279742da [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fd89afa0150a4437b4f2ea95593ccaf0 (127.25.92.195:41629), 69b377dded7945329549bf5a64bf87b1 (127.25.92.194:44511)
W20260805 23:14:09.466920 10613 leader_election.cc:341] T 748bd38edc9844d3941a5797032ce70d P 85d51d37bc304498873dd204279742da [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 69b377dded7945329549bf5a64bf87b1 (127.25.92.194:44511): Not found: Tablet not found: 748bd38edc9844d3941a5797032ce70d
W20260805 23:14:09.467098 10611 leader_election.cc:341] T 748bd38edc9844d3941a5797032ce70d P 85d51d37bc304498873dd204279742da [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fd89afa0150a4437b4f2ea95593ccaf0 (127.25.92.195:41629): Not found: Tablet not found: 748bd38edc9844d3941a5797032ce70d
I20260805 23:14:09.467190 10611 leader_election.cc:302] T 748bd38edc9844d3941a5797032ce70d P 85d51d37bc304498873dd204279742da [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85d51d37bc304498873dd204279742da; no voters: 69b377dded7945329549bf5a64bf87b1, fd89afa0150a4437b4f2ea95593ccaf0
I20260805 23:14:09.467406 11025 raft_consensus.cc:2726] T 748bd38edc9844d3941a5797032ce70d P 85d51d37bc304498873dd204279742da [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260805 23:14:09.531136 11026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:09.530695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:34379 (local address 127.25.92.193:37401)
0805 23:14:09.530962 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:09.531033 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:34379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":80,"threads_started":1}
I20260805 23:14:09.694600 11027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:09.694075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:34379 (local address 127.25.92.194:45719)
0805 23:14:09.694397 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:09.694456 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:34379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":94,"threads_started":1}
I20260805 23:14:09.891340 11029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:09.890855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:34379 (local address 127.25.92.195:49791)
0805 23:14:09.891161 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:09.891238 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:34379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":94,"threads_started":1}
I20260805 23:14:10.532575 11030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:10.532082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:34379 (local address 127.25.92.193:34843)
0805 23:14:10.532389 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:10.532463 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:34379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":77,"threads_started":1}
I20260805 23:14:10.696004 11031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:10.695499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:34379 (local address 127.25.92.194:36139)
0805 23:14:10.695837 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:10.695901 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:34379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":146,"threads_started":1}
I20260805 23:14:10.892622 11032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:10.892174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:34379 (local address 127.25.92.195:40021)
0805 23:14:10.892454 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:10.892511 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:34379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":93,"threads_started":1}
I20260805 23:14:11.534336 11033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:11.533845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:34379 (local address 127.25.92.193:60791)
0805 23:14:11.534150 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:11.534235 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:34379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":74,"threads_started":1}
I20260805 23:14:11.584545 11034 raft_consensus.cc:482] T 748bd38edc9844d3941a5797032ce70d P 85d51d37bc304498873dd204279742da [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 23:14:11.584686 11034 raft_consensus.cc:504] T 748bd38edc9844d3941a5797032ce70d P 85d51d37bc304498873dd204279742da [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd89afa0150a4437b4f2ea95593ccaf0" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 41629 } } peers { permanent_uuid: "69b377dded7945329549bf5a64bf87b1" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 44511 } } peers { permanent_uuid: "85d51d37bc304498873dd204279742da" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 45305 } }
I20260805 23:14:11.585181 11034 leader_election.cc:288] T 748bd38edc9844d3941a5797032ce70d P 85d51d37bc304498873dd204279742da [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fd89afa0150a4437b4f2ea95593ccaf0 (127.25.92.195:41629), 69b377dded7945329549bf5a64bf87b1 (127.25.92.194:44511)
W20260805 23:14:11.586122 10613 leader_election.cc:341] T 748bd38edc9844d3941a5797032ce70d P 85d51d37bc304498873dd204279742da [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 69b377dded7945329549bf5a64bf87b1 (127.25.92.194:44511): Not found: Tablet not found: 748bd38edc9844d3941a5797032ce70d
W20260805 23:14:11.586216 10611 leader_election.cc:341] T 748bd38edc9844d3941a5797032ce70d P 85d51d37bc304498873dd204279742da [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fd89afa0150a4437b4f2ea95593ccaf0 (127.25.92.195:41629): Not found: Tablet not found: 748bd38edc9844d3941a5797032ce70d
I20260805 23:14:11.586325 10611 leader_election.cc:302] T 748bd38edc9844d3941a5797032ce70d P 85d51d37bc304498873dd204279742da [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85d51d37bc304498873dd204279742da; no voters: 69b377dded7945329549bf5a64bf87b1, fd89afa0150a4437b4f2ea95593ccaf0
I20260805 23:14:11.586488 11034 raft_consensus.cc:2726] T 748bd38edc9844d3941a5797032ce70d P 85d51d37bc304498873dd204279742da [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260805 23:14:11.697547 11035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:11.697013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:34379 (local address 127.25.92.194:60483)
0805 23:14:11.697323 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:11.697394 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:34379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":83,"threads_started":1}
I20260805 23:14:11.894032 11036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:11.893588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:34379 (local address 127.25.92.195:42183)
0805 23:14:11.893873 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:11.893932 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:34379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":101,"threads_started":1}
I20260805 23:14:12.535775 11037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:12.535272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:34379 (local address 127.25.92.193:40563)
0805 23:14:12.535574 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:12.535675 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:34379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":101,"threads_started":1}
I20260805 23:14:12.699054 11038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:12.698580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:34379 (local address 127.25.92.194:52563)
0805 23:14:12.698866 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:12.698940 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:34379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":88,"threads_started":1}
I20260805 23:14:12.895318 11039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:12.894877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:34379 (local address 127.25.92.195:57799)
0805 23:14:12.895164 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:12.895220 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:34379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":99,"threads_started":1}
I20260805 23:14:13.537108 11041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:13.536625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:34379 (local address 127.25.92.193:34443)
0805 23:14:13.536923 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:13.536993 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:34379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":81,"threads_started":1}
I20260805 23:14:13.700670 11042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:13.700039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:34379 (local address 127.25.92.194:43517)
0805 23:14:13.700402 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:13.700545 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:34379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":92,"threads_started":1}
I20260805 23:14:13.869680 11043 raft_consensus.cc:482] T 748bd38edc9844d3941a5797032ce70d P 85d51d37bc304498873dd204279742da [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 23:14:13.869822 11043 raft_consensus.cc:504] T 748bd38edc9844d3941a5797032ce70d P 85d51d37bc304498873dd204279742da [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd89afa0150a4437b4f2ea95593ccaf0" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 41629 } } peers { permanent_uuid: "69b377dded7945329549bf5a64bf87b1" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 44511 } } peers { permanent_uuid: "85d51d37bc304498873dd204279742da" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 45305 } }
I20260805 23:14:13.870304 11043 leader_election.cc:288] T 748bd38edc9844d3941a5797032ce70d P 85d51d37bc304498873dd204279742da [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fd89afa0150a4437b4f2ea95593ccaf0 (127.25.92.195:41629), 69b377dded7945329549bf5a64bf87b1 (127.25.92.194:44511)
W20260805 23:14:13.871326 10613 leader_election.cc:341] T 748bd38edc9844d3941a5797032ce70d P 85d51d37bc304498873dd204279742da [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 69b377dded7945329549bf5a64bf87b1 (127.25.92.194:44511): Not found: Tablet not found: 748bd38edc9844d3941a5797032ce70d
W20260805 23:14:13.871438 10611 leader_election.cc:341] T 748bd38edc9844d3941a5797032ce70d P 85d51d37bc304498873dd204279742da [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fd89afa0150a4437b4f2ea95593ccaf0 (127.25.92.195:41629): Not found: Tablet not found: 748bd38edc9844d3941a5797032ce70d
I20260805 23:14:13.871588 10611 leader_election.cc:302] T 748bd38edc9844d3941a5797032ce70d P 85d51d37bc304498873dd204279742da [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85d51d37bc304498873dd204279742da; no voters: 69b377dded7945329549bf5a64bf87b1, fd89afa0150a4437b4f2ea95593ccaf0
I20260805 23:14:13.871738 11043 raft_consensus.cc:2726] T 748bd38edc9844d3941a5797032ce70d P 85d51d37bc304498873dd204279742da [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260805 23:14:13.896741 11044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:13.896199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:34379 (local address 127.25.92.195:37961)
0805 23:14:13.896543 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:13.896599 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:34379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":98,"threads_started":1}
I20260805 23:14:14.538703 11045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:14.538154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:34379 (local address 127.25.92.193:59627)
0805 23:14:14.538456 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:14.538575 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:34379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":82,"threads_started":1}
I20260805 23:14:14.702332 11046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:14.701716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:34379 (local address 127.25.92.194:46597)
0805 23:14:14.702060 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:14.702132 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:34379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":130,"threads_started":1}
I20260805 23:14:14.898308 11047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:14.897755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:34379 (local address 127.25.92.195:34135)
0805 23:14:14.898133 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:14.898200 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:34379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":107,"threads_started":1}
I20260805 23:14:15.540202 11048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:15.539668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:34379 (local address 127.25.92.193:56153)
0805 23:14:15.540012 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:15.540084 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:34379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":77,"threads_started":1}
I20260805 23:14:15.703704 11049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:15.703227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:34379 (local address 127.25.92.194:42869)
0805 23:14:15.703542 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:15.703601 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:34379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":114,"threads_started":1}
I20260805 23:14:15.900169 11051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:15.899627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:34379 (local address 127.25.92.195:57923)
0805 23:14:15.899983 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:15.900060 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:34379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":121,"threads_started":1}
I20260805 23:14:16.541878 11052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:16.541330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:34379 (local address 127.25.92.193:56223)
0805 23:14:16.541709 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:16.541777 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:34379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":103,"threads_started":1}
I20260805 23:14:16.705302 11053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:16.704816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:34379 (local address 127.25.92.194:51365)
0805 23:14:16.705124 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:16.705200 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:34379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":86,"threads_started":1}
I20260805 23:14:16.901778 11054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:16.901203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:34379 (local address 127.25.92.195:58689)
0805 23:14:16.901477 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:16.901669 (+ 192us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:34379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":99,"threads_started":1}
I20260805 23:14:17.543368 11055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:17.542846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:34379 (local address 127.25.92.193:39797)
0805 23:14:17.543200 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:17.543267 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:34379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":93,"threads_started":1}
I20260805 23:14:17.706764 11056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:17.706283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:34379 (local address 127.25.92.194:49519)
0805 23:14:17.706585 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:17.706657 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:34379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":79,"threads_started":1}
I20260805 23:14:17.903429 11057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:17.902762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:34379 (local address 127.25.92.195:49805)
0805 23:14:17.903203 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:17.903299 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:34379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":100,"threads_started":1}
I20260805 23:14:18.534605 11058 raft_consensus.cc:482] T 748bd38edc9844d3941a5797032ce70d P 85d51d37bc304498873dd204279742da [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 23:14:18.534749 11058 raft_consensus.cc:504] T 748bd38edc9844d3941a5797032ce70d P 85d51d37bc304498873dd204279742da [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd89afa0150a4437b4f2ea95593ccaf0" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 41629 } } peers { permanent_uuid: "69b377dded7945329549bf5a64bf87b1" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 44511 } } peers { permanent_uuid: "85d51d37bc304498873dd204279742da" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 45305 } }
I20260805 23:14:18.535197 11058 leader_election.cc:288] T 748bd38edc9844d3941a5797032ce70d P 85d51d37bc304498873dd204279742da [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fd89afa0150a4437b4f2ea95593ccaf0 (127.25.92.195:41629), 69b377dded7945329549bf5a64bf87b1 (127.25.92.194:44511)
W20260805 23:14:18.536077 10611 leader_election.cc:341] T 748bd38edc9844d3941a5797032ce70d P 85d51d37bc304498873dd204279742da [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fd89afa0150a4437b4f2ea95593ccaf0 (127.25.92.195:41629): Not found: Tablet not found: 748bd38edc9844d3941a5797032ce70d
W20260805 23:14:18.536185 10613 leader_election.cc:341] T 748bd38edc9844d3941a5797032ce70d P 85d51d37bc304498873dd204279742da [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 69b377dded7945329549bf5a64bf87b1 (127.25.92.194:44511): Not found: Tablet not found: 748bd38edc9844d3941a5797032ce70d
I20260805 23:14:18.536312 10613 leader_election.cc:302] T 748bd38edc9844d3941a5797032ce70d P 85d51d37bc304498873dd204279742da [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 85d51d37bc304498873dd204279742da; no voters: 69b377dded7945329549bf5a64bf87b1, fd89afa0150a4437b4f2ea95593ccaf0
I20260805 23:14:18.536499 11058 raft_consensus.cc:2726] T 748bd38edc9844d3941a5797032ce70d P 85d51d37bc304498873dd204279742da [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260805 23:14:18.544816 11059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:18.544385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:34379 (local address 127.25.92.193:48605)
0805 23:14:18.544645 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:18.544705 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:34379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":79,"threads_started":1}
I20260805 23:14:18.708153 11060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:18.707694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:34379 (local address 127.25.92.194:48001)
0805 23:14:18.707979 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:18.708050 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:34379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":84,"threads_started":1}
I20260805 23:14:18.905233 11061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:18.904491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:34379 (local address 127.25.92.195:34917)
0805 23:14:18.904978 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:18.905104 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:34379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":164,"threads_started":1}
I20260805 23:14:19.546331 11062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:19.545840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:34379 (local address 127.25.92.193:58683)
0805 23:14:19.546163 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:19.546233 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:34379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":79,"threads_started":1}
I20260805 23:14:19.709844 11063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:19.709129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:34379 (local address 127.25.92.194:52151)
0805 23:14:19.709471 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:19.709724 (+ 253us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:34379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":113,"threads_started":1}
I20260805 23:14:19.906662 11064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:19.906196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:34379 (local address 127.25.92.195:49889)
0805 23:14:19.906491 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:19.906554 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:34379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":97,"threads_started":1}
I20260805 23:14:20.548098 11065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:20.547525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:34379 (local address 127.25.92.193:50415)
0805 23:14:20.547911 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:20.547986 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:34379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":146,"threads_started":1}
I20260805 23:14:20.711427 11066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:20.710835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:34379 (local address 127.25.92.194:55725)
0805 23:14:20.711246 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:20.711324 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:34379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":85,"threads_started":1}
I20260805 23:14:20.908005 11067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:20.907571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:34379 (local address 127.25.92.195:43679)
0805 23:14:20.907846 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:20.907905 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:34379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":100,"threads_started":1}
I20260805 23:14:21.549803 11068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:21.549223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:34379 (local address 127.25.92.193:40581)
0805 23:14:21.549600 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:21.549702 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:34379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":78,"threads_started":1}
I20260805 23:14:21.713001 11069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:21.712513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:34379 (local address 127.25.92.194:38857)
0805 23:14:21.712801 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:21.712863 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.254:34379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":96,"threads_started":1}
I20260805 23:14:21.891285 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 10551
I20260805 23:14:21.898012 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 10733
I20260805 23:14:21.904649 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 10875
2026-08-05T23:14:21Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59524 ms)
[----------] 30 tests from SecurityITest (255622 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971406251336-25971-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 23:14:21 dist-test-slave-ss4n krb5kdc[11072](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 23:14:21 dist-test-slave-ss4n krb5kdc[11072](info): set up 2 sockets
Aug 05 23:14:21 dist-test-slave-ss4n krb5kdc[11072](info): commencing operation
krb5kdc: starting...
W20260805 23:14:23.013154 25971 mini_kdc.cc:121] Time spent starting KDC: real 1.084s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971406251336-25971-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971406251336-25971-0/krb5kdc/test-user.keytab.
Aug 05 23:14:23 dist-test-slave-ss4n krb5kdc[11072](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971663, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T23:14:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:14:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.92.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.92.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.92.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab.
Entry for principal kudu/127.25.92.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab.
I20260805 23:14:23.162274 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971406251336-25971-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971406251336-25971-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971406251336-25971-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.254:46225
--webserver_interface=127.25.92.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab
--principal=kudu/127.25.92.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:40265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.92.254:46225
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:14:23.281260 11085 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:14:23.281695 11085 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:14:23.281788 11085 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:14:23.286399 11085 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:14:23.286517 11085 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:14:23.286564 11085 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:14:23.286605 11085 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:14:23.286641 11085 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:14:23.293267 11085 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:40265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971406251336-25971-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971406251336-25971-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.92.254:46225
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab
--principal=kudu/127.25.92.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.254:46225
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.92.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11085
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971406251336-25971-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:14:23.294679 11085 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:14:23.295931 11085 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:14:23.307608 11091 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:14:23.307672 11093 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:14:23.307726 11090 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:14:23.307790 11085 server_base.cc:1034] running on GCE node
I20260805 23:14:23.308738 11085 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:14:23.310197 11085 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:14:23.311424 11085 hybrid_clock.cc:648] HybridClock initialized: now 1785971663311388 us; error 55 us; skew 500 ppm
Aug 05 23:14:23 dist-test-slave-ss4n krb5kdc[11072](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971663, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:14:23.319233 11085 init.cc:377] Logged in from keytab as kudu/127.25.92.254@KRBTEST.COM (short username kudu)
I20260805 23:14:23.320428 11085 webserver.cc:460] Webserver started at http://127.25.92.254:41897/ using document root <none> and password file <none>
I20260805 23:14:23.321049 11085 fs_manager.cc:359] Metadata directory not provided
I20260805 23:14:23.321125 11085 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:14:23.321321 11085 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:14:23.323832 11085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971406251336-25971-0/minicluster-data/master-0/data/instance:
uuid: "10c45b269fa448c4af9996257a81341e"
format_stamp: "Formatted at 2026-08-05 23:14:23 on dist-test-slave-ss4n"
server_key: "59e0ba309a3dc580065767fac5ab81c2"
server_key_iv: "7d57a2cf622ef59164a389909ead53bd"
server_key_version: "encryptionkey@0"
I20260805 23:14:23.324469 11085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971406251336-25971-0/minicluster-data/master-0/wal/instance:
uuid: "10c45b269fa448c4af9996257a81341e"
format_stamp: "Formatted at 2026-08-05 23:14:23 on dist-test-slave-ss4n"
server_key: "59e0ba309a3dc580065767fac5ab81c2"
server_key_iv: "7d57a2cf622ef59164a389909ead53bd"
server_key_version: "encryptionkey@0"
I20260805 23:14:23.329178 11085 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.005s
I20260805 23:14:23.332301 11101 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:14:23.333890 11085 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260805 23:14:23.334019 11085 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971406251336-25971-0/minicluster-data/master-0/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971406251336-25971-0/minicluster-data/master-0/wal
uuid: "10c45b269fa448c4af9996257a81341e"
format_stamp: "Formatted at 2026-08-05 23:14:23 on dist-test-slave-ss4n"
server_key: "59e0ba309a3dc580065767fac5ab81c2"
server_key_iv: "7d57a2cf622ef59164a389909ead53bd"
server_key_version: "encryptionkey@0"
I20260805 23:14:23.334139 11085 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971406251336-25971-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971406251336-25971-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971406251336-25971-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:14:23.354445 11085 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:14:23.358624 11085 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:14:23.358840 11085 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:14:23.368671 11085 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.254:46225
I20260805 23:14:23.368798 11155 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.254:46225 every 8 connection(s)
I20260805 23:14:23.369937 11085 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
I20260805 23:14:23.371071 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 11085
I20260805 23:14:23.371212 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971406251336-25971-0/minicluster-data/master-0/wal/instance
I20260805 23:14:23.371519 25971 external_mini_cluster.cc:1428] Setting key 73ca901ab017efaa2c7d4dd0ef81abe8
I20260805 23:14:23.374259 11156 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:14:23.382035 11156 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10c45b269fa448c4af9996257a81341e: Bootstrap starting.
Aug 05 23:14:23 dist-test-slave-ss4n krb5kdc[11072](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971663, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM
I20260805 23:14:23.385334 11156 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 10c45b269fa448c4af9996257a81341e: Neither blocks nor log segments found. Creating new log.
I20260805 23:14:23.386423 11156 log.cc:824] T 00000000000000000000000000000000 P 10c45b269fa448c4af9996257a81341e: Log is configured to *not* fsync() on all Append() calls
I20260805 23:14:23.389070 11156 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10c45b269fa448c4af9996257a81341e: No bootstrap required, opened a new log
I20260805 23:14:23.389412 11159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:23.373072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53300 (local address 127.25.92.254:46225)
0805 23:14:23.373596 (+ 524us) server_negotiation.cc:206] Beginning negotiation
0805 23:14:23.373622 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0805 23:14:23.373669 (+ 47us) server_negotiation.cc:407] Connection header received
0805 23:14:23.374779 (+ 1110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:14:23.374800 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:14:23.375169 (+ 369us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:14:23.375504 (+ 335us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:14:23.376600 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:23.377944 (+ 1344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:23.379268 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:23.379870 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:23.380008 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:23.383339 (+ 3331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:14:23.383348 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:14:23.383369 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:14:23.383414 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:14:23.385934 (+ 2520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:14:23.386445 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:14:23.386453 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:14:23.386459 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:14:23.386618 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:14:23.386936 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:14:23.386939 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:14:23.386943 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:14:23.387443 (+ 500us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:14:23.387655 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0805 23:14:23.387985 (+ 330us) server_negotiation.cc:299] Negotiation successful
0805 23:14:23.388331 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326,"thread_start_us":166,"threads_started":1}
I20260805 23:14:23.393643 11156 raft_consensus.cc:348] T 00000000000000000000000000000000 P 10c45b269fa448c4af9996257a81341e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10c45b269fa448c4af9996257a81341e" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 46225 } }
I20260805 23:14:23.393903 11156 raft_consensus.cc:374] T 00000000000000000000000000000000 P 10c45b269fa448c4af9996257a81341e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:14:23.393942 11156 raft_consensus.cc:729] T 00000000000000000000000000000000 P 10c45b269fa448c4af9996257a81341e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10c45b269fa448c4af9996257a81341e, State: Initialized, Role: FOLLOWER
I20260805 23:14:23.394410 11156 consensus_queue.cc:260] T 00000000000000000000000000000000 P 10c45b269fa448c4af9996257a81341e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10c45b269fa448c4af9996257a81341e" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 46225 } }
I20260805 23:14:23.394629 11156 raft_consensus.cc:388] T 00000000000000000000000000000000 P 10c45b269fa448c4af9996257a81341e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:14:23.394695 11156 raft_consensus.cc:482] T 00000000000000000000000000000000 P 10c45b269fa448c4af9996257a81341e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:14:23.394780 11156 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 10c45b269fa448c4af9996257a81341e [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:14:23.395829 11156 raft_consensus.cc:504] T 00000000000000000000000000000000 P 10c45b269fa448c4af9996257a81341e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10c45b269fa448c4af9996257a81341e" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 46225 } }
I20260805 23:14:23.396170 11156 leader_election.cc:302] T 00000000000000000000000000000000 P 10c45b269fa448c4af9996257a81341e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 10c45b269fa448c4af9996257a81341e; no voters:
I20260805 23:14:23.396466 11156 leader_election.cc:288] T 00000000000000000000000000000000 P 10c45b269fa448c4af9996257a81341e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 23:14:23.396744 11161 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 10c45b269fa448c4af9996257a81341e [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:14:23.397044 11161 raft_consensus.cc:686] T 00000000000000000000000000000000 P 10c45b269fa448c4af9996257a81341e [term 1 LEADER]: Becoming Leader. State: Replica: 10c45b269fa448c4af9996257a81341e, State: Running, Role: LEADER
I20260805 23:14:23.397380 11161 consensus_queue.cc:237] T 00000000000000000000000000000000 P 10c45b269fa448c4af9996257a81341e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10c45b269fa448c4af9996257a81341e" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 46225 } }
I20260805 23:14:23.398070 11156 sys_catalog.cc:564] T 00000000000000000000000000000000 P 10c45b269fa448c4af9996257a81341e [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:14:23.399458 11163 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10c45b269fa448c4af9996257a81341e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 10c45b269fa448c4af9996257a81341e. Latest consensus state: current_term: 1 leader_uuid: "10c45b269fa448c4af9996257a81341e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10c45b269fa448c4af9996257a81341e" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 46225 } } }
I20260805 23:14:23.399623 11163 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10c45b269fa448c4af9996257a81341e [sys.catalog]: This master's current role is: LEADER
I20260805 23:14:23.399458 11162 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10c45b269fa448c4af9996257a81341e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "10c45b269fa448c4af9996257a81341e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10c45b269fa448c4af9996257a81341e" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 46225 } } }
I20260805 23:14:23.399705 11162 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10c45b269fa448c4af9996257a81341e [sys.catalog]: This master's current role is: LEADER
I20260805 23:14:23.400775 11172 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:14:23.403805 11172 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:14:23.410290 11172 catalog_manager.cc:1347] Generated new cluster ID: 54fd002a19c14aabac3c53949f96601c
I20260805 23:14:23.410375 11172 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:14:23.418021 11172 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:14:23.419003 11172 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:14:23.430260 11172 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 10c45b269fa448c4af9996257a81341e: Generated new TSK 0
I20260805 23:14:23.431270 11172 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.92.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.92.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.92.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.193.keytab.
Entry for principal kudu/127.25.92.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.193.keytab.
I20260805 23:14:23.477058 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971406251336-25971-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971406251336-25971-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971406251336-25971-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971406251336-25971-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.92.193:0
--local_ip_for_outbound_sockets=127.25.92.193
--webserver_interface=127.25.92.193
--webserver_port=0
--tserver_master_addrs=127.25.92.254:46225
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.193.keytab
--principal=kudu/127.25.92.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:40265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:14:23.608541 11184 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:14:23.608860 11184 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:14:23.608907 11184 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:14:23.613587 11184 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:14:23.613680 11184 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:14:23.613747 11184 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.92.193
I20260805 23:14:23.620615 11184 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:40265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971406251336-25971-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971406251336-25971-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.193.keytab
--principal=kudu/127.25.92.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.92.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971406251336-25971-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.92.193
--webserver_port=0
--tserver_master_addrs=127.25.92.254:46225
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11184
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.92.193
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971406251336-25971-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:14:23.621994 11184 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:14:23.623230 11184 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:14:23.634938 11190 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:14:23.634971 11192 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:14:23.634959 11189 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:14:23.635332 11184 server_base.cc:1034] running on GCE node
I20260805 23:14:23.635960 11184 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:14:23.637254 11184 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:14:23.638533 11184 hybrid_clock.cc:648] HybridClock initialized: now 1785971663638492 us; error 71 us; skew 500 ppm
Aug 05 23:14:23 dist-test-slave-ss4n krb5kdc[11072](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971663, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:14:23.642272 11184 init.cc:377] Logged in from keytab as kudu/127.25.92.193@KRBTEST.COM (short username kudu)
I20260805 23:14:23.643539 11184 webserver.cc:460] Webserver started at http://127.25.92.193:42971/ using document root <none> and password file <none>
I20260805 23:14:23.644214 11184 fs_manager.cc:359] Metadata directory not provided
I20260805 23:14:23.644335 11184 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:14:23.644963 11184 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:14:23.647958 11184 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971406251336-25971-0/minicluster-data/ts-0/data/instance:
uuid: "3520975d65424a2f90b8bd1d59ac27a3"
format_stamp: "Formatted at 2026-08-05 23:14:23 on dist-test-slave-ss4n"
server_key: "f4c01c7c6b62e49e69ebec4a3be43d92"
server_key_iv: "0b8d510650e41ad460d1da9581e9e0e0"
server_key_version: "encryptionkey@0"
I20260805 23:14:23.648720 11184 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971406251336-25971-0/minicluster-data/ts-0/wal/instance:
uuid: "3520975d65424a2f90b8bd1d59ac27a3"
format_stamp: "Formatted at 2026-08-05 23:14:23 on dist-test-slave-ss4n"
server_key: "f4c01c7c6b62e49e69ebec4a3be43d92"
server_key_iv: "0b8d510650e41ad460d1da9581e9e0e0"
server_key_version: "encryptionkey@0"
I20260805 23:14:23.653456 11184 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260805 23:14:23.656795 11199 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:14:23.658398 11184 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:14:23.658587 11184 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971406251336-25971-0/minicluster-data/ts-0/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971406251336-25971-0/minicluster-data/ts-0/wal
uuid: "3520975d65424a2f90b8bd1d59ac27a3"
format_stamp: "Formatted at 2026-08-05 23:14:23 on dist-test-slave-ss4n"
server_key: "f4c01c7c6b62e49e69ebec4a3be43d92"
server_key_iv: "0b8d510650e41ad460d1da9581e9e0e0"
server_key_version: "encryptionkey@0"
I20260805 23:14:23.658737 11184 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971406251336-25971-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971406251336-25971-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971406251336-25971-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:14:23.681198 11184 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:14:23.685019 11184 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:14:23.685256 11184 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:14:23.686021 11184 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:14:23.687007 11184 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:14:23.687086 11184 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:14:23.687161 11184 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:14:23.687208 11184 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:14:23.697474 11184 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.193:32955
I20260805 23:14:23.697530 11317 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.193:32955 every 8 connection(s)
I20260805 23:14:23.698787 11184 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971406251336-25971-0/minicluster-data/ts-0/data/info.pb
I20260805 23:14:23.706146 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 11184
I20260805 23:14:23.706292 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971406251336-25971-0/minicluster-data/ts-0/wal/instance
I20260805 23:14:23.706597 25971 external_mini_cluster.cc:1428] Setting key deea36564148ceb443c1c66011ce17b8
Aug 05 23:14:23 dist-test-slave-ss4n krb5kdc[11072](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971663, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.193@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM
I20260805 23:14:23.716459 11159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:23.700914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.193:37769 (local address 127.25.92.254:46225)
0805 23:14:23.701073 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0805 23:14:23.701077 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:14:23.702315 (+ 1238us) server_negotiation.cc:407] Connection header received
0805 23:14:23.703876 (+ 1561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:14:23.703880 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:14:23.703946 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:14:23.704042 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:14:23.705419 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:23.706313 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:23.707929 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:23.708213 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:23.708350 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:23.711413 (+ 3063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:14:23.711417 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:14:23.711419 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:14:23.711454 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:14:23.713248 (+ 1794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:14:23.714161 (+ 913us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:14:23.714164 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:14:23.714166 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:14:23.714256 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:14:23.714825 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:14:23.714833 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:14:23.714836 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:14:23.715139 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:14:23.715245 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0805 23:14:23.716143 (+ 898us) server_negotiation.cc:299] Negotiation successful
0805 23:14:23.716305 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260805 23:14:23.717875 11320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:23.701292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:46225 (local address 127.25.92.193:37769)
0805 23:14:23.702061 (+ 769us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:23.702113 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0805 23:14:23.703610 (+ 1497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:14:23.704251 (+ 641us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:14:23.704259 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:14:23.704680 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:14:23.705237 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:14:23.705254 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:14:23.706515 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:14:23.706519 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:14:23.707794 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:14:23.707806 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:14:23.708404 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:14:23.708409 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:14:23.708602 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:23.709302 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:14:23.709319 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:14:23.711139 (+ 1820us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:14:23.713412 (+ 2273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:14:23.713424 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:14:23.713451 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:14:23.713993 (+ 542us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:14:23.714378 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:14:23.714385 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:14:23.714389 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:14:23.714645 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:14:23.715338 (+ 693us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:14:23.715348 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:14:23.715789 (+ 441us) client_negotiation.cc:769] Sending connection context
0805 23:14:23.716050 (+ 261us) client_negotiation.cc:240] Negotiation successful
0805 23:14:23.716501 (+ 451us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":523,"thread_start_us":149,"threads_started":1}
WARNING: no policy specified for kudu/127.25.92.194@KRBTEST.COM; defaulting to no policy
I20260805 23:14:23.719660 11318 heartbeater.cc:347] Connected to a master server at 127.25.92.254:46225
I20260805 23:14:23.719918 11318 heartbeater.cc:464] Registering TS with master...
I20260805 23:14:23.720438 11318 heartbeater.cc:511] Master 127.25.92.254:46225 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.92.194@KRBTEST.COM" created.
I20260805 23:14:23.722213 11118 ts_manager.cc:194] Registered new tserver with Master: 3520975d65424a2f90b8bd1d59ac27a3 (127.25.92.193:32955)
I20260805 23:14:23.723995 11118 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.92.193@KRBTEST.COM'} at 127.25.92.193:37769
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.92.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.194.keytab.
Entry for principal kudu/127.25.92.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.194.keytab.
I20260805 23:14:23.741920 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971406251336-25971-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971406251336-25971-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971406251336-25971-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971406251336-25971-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.92.194:0
--local_ip_for_outbound_sockets=127.25.92.194
--webserver_interface=127.25.92.194
--webserver_port=0
--tserver_master_addrs=127.25.92.254:46225
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.194.keytab
--principal=kudu/127.25.92.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:40265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:14:23.865722 11323 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:14:23.866034 11323 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:14:23.866080 11323 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:14:23.870509 11323 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:14:23.870895 11323 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:14:23.870997 11323 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.92.194
I20260805 23:14:23.876869 11323 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:40265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971406251336-25971-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971406251336-25971-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.194.keytab
--principal=kudu/127.25.92.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.92.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971406251336-25971-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.92.194
--webserver_port=0
--tserver_master_addrs=127.25.92.254:46225
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11323
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.92.194
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971406251336-25971-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:14:23.878047 11323 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:14:23.879141 11323 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:14:23.890048 11331 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:14:23.890026 11329 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:14:23.890062 11328 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:14:23.890295 11323 server_base.cc:1034] running on GCE node
I20260805 23:14:23.890858 11323 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:14:23.891660 11323 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:14:23.892894 11323 hybrid_clock.cc:648] HybridClock initialized: now 1785971663892858 us; error 52 us; skew 500 ppm
Aug 05 23:14:23 dist-test-slave-ss4n krb5kdc[11072](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971663, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:14:23.896451 11323 init.cc:377] Logged in from keytab as kudu/127.25.92.194@KRBTEST.COM (short username kudu)
I20260805 23:14:23.897583 11323 webserver.cc:460] Webserver started at http://127.25.92.194:33021/ using document root <none> and password file <none>
I20260805 23:14:23.898253 11323 fs_manager.cc:359] Metadata directory not provided
I20260805 23:14:23.898350 11323 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:14:23.898626 11323 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:14:23.901069 11323 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971406251336-25971-0/minicluster-data/ts-1/data/instance:
uuid: "28bf3bd95d17438e8a3669056f58c8fd"
format_stamp: "Formatted at 2026-08-05 23:14:23 on dist-test-slave-ss4n"
server_key: "beb1cf66d0a5dee17672172bc5410089"
server_key_iv: "7e1b75c82c6bbd44adf56f52804034cc"
server_key_version: "encryptionkey@0"
I20260805 23:14:23.901750 11323 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971406251336-25971-0/minicluster-data/ts-1/wal/instance:
uuid: "28bf3bd95d17438e8a3669056f58c8fd"
format_stamp: "Formatted at 2026-08-05 23:14:23 on dist-test-slave-ss4n"
server_key: "beb1cf66d0a5dee17672172bc5410089"
server_key_iv: "7e1b75c82c6bbd44adf56f52804034cc"
server_key_version: "encryptionkey@0"
I20260805 23:14:23.906015 11323 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260805 23:14:23.908668 11338 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:14:23.910254 11323 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:14:23.910416 11323 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971406251336-25971-0/minicluster-data/ts-1/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971406251336-25971-0/minicluster-data/ts-1/wal
uuid: "28bf3bd95d17438e8a3669056f58c8fd"
format_stamp: "Formatted at 2026-08-05 23:14:23 on dist-test-slave-ss4n"
server_key: "beb1cf66d0a5dee17672172bc5410089"
server_key_iv: "7e1b75c82c6bbd44adf56f52804034cc"
server_key_version: "encryptionkey@0"
I20260805 23:14:23.910557 11323 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971406251336-25971-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971406251336-25971-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971406251336-25971-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:14:23.936265 11323 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:14:23.940594 11323 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:14:23.940840 11323 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:14:23.941576 11323 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:14:23.942548 11323 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:14:23.942636 11323 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:14:23.942735 11323 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:14:23.942790 11323 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:14:23.953831 11323 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.194:46869
I20260805 23:14:23.953909 11456 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.194:46869 every 8 connection(s)
I20260805 23:14:23.954969 11323 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971406251336-25971-0/minicluster-data/ts-1/data/info.pb
I20260805 23:14:23.960748 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 11323
I20260805 23:14:23.960889 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971406251336-25971-0/minicluster-data/ts-1/wal/instance
I20260805 23:14:23.961261 25971 external_mini_cluster.cc:1428] Setting key 949be54cfa8ff4cb5c583d01ef6b2aa3
Aug 05 23:14:23 dist-test-slave-ss4n krb5kdc[11072](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971663, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.194@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM
I20260805 23:14:23.972226 11159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:23.957301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.194:48403 (local address 127.25.92.254:46225)
0805 23:14:23.957553 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0805 23:14:23.957560 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0805 23:14:23.958598 (+ 1038us) server_negotiation.cc:407] Connection header received
0805 23:14:23.960234 (+ 1636us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:14:23.960239 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:14:23.960309 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:14:23.960440 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:14:23.962264 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:23.962974 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:23.964407 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:23.964654 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:23.964781 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:23.967625 (+ 2844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:14:23.967629 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:14:23.967631 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:14:23.967667 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:14:23.969717 (+ 2050us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:14:23.970381 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:14:23.970386 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:14:23.970390 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:14:23.970484 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:14:23.970916 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:14:23.970920 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:14:23.970922 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:14:23.971156 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:14:23.971281 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0805 23:14:23.971879 (+ 598us) server_negotiation.cc:299] Negotiation successful
0805 23:14:23.972036 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
WARNING: no policy specified for kudu/127.25.92.195@KRBTEST.COM; defaulting to no policy
I20260805 23:14:23.973820 11459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:23.957829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:46225 (local address 127.25.92.194:48403)
0805 23:14:23.958406 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:23.958458 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0805 23:14:23.959969 (+ 1511us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:14:23.960652 (+ 683us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:14:23.960668 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:14:23.961252 (+ 584us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:14:23.962063 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:14:23.962082 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:14:23.963108 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:14:23.963112 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:14:23.964249 (+ 1137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:14:23.964261 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:14:23.964838 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:14:23.964843 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:14:23.965039 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:23.965778 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:14:23.965796 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:14:23.967400 (+ 1604us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:14:23.969876 (+ 2476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:14:23.969884 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:14:23.969906 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:14:23.970248 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:14:23.970616 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:14:23.970620 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:14:23.970622 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:14:23.970798 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:14:23.971281 (+ 483us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:14:23.971291 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:14:23.971604 (+ 313us) client_negotiation.cc:769] Sending connection context
0805 23:14:23.971818 (+ 214us) client_negotiation.cc:240] Negotiation successful
0805 23:14:23.972229 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":158,"threads_started":1}
I20260805 23:14:23.976016 11457 heartbeater.cc:347] Connected to a master server at 127.25.92.254:46225
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.92.195@KRBTEST.COM" created.
I20260805 23:14:23.976477 11457 heartbeater.cc:464] Registering TS with master...
I20260805 23:14:23.977593 11457 heartbeater.cc:511] Master 127.25.92.254:46225 requested a full tablet report, sending...
I20260805 23:14:23.979184 11118 ts_manager.cc:194] Registered new tserver with Master: 28bf3bd95d17438e8a3669056f58c8fd (127.25.92.194:46869)
I20260805 23:14:23.979871 11118 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.92.194@KRBTEST.COM'} at 127.25.92.194:48403
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.92.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.195.keytab.
Entry for principal kudu/127.25.92.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.195.keytab.
I20260805 23:14:23.997107 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971406251336-25971-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971406251336-25971-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971406251336-25971-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971406251336-25971-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.92.195:0
--local_ip_for_outbound_sockets=127.25.92.195
--webserver_interface=127.25.92.195
--webserver_port=0
--tserver_master_addrs=127.25.92.254:46225
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.195.keytab
--principal=kudu/127.25.92.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:40265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:14:24.121328 11462 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:14:24.121701 11462 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:14:24.121783 11462 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:14:24.126216 11462 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:14:24.126334 11462 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:14:24.126451 11462 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.92.195
I20260805 23:14:24.132181 11462 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:40265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971406251336-25971-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971406251336-25971-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.195.keytab
--principal=kudu/127.25.92.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.92.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971406251336-25971-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.92.195
--webserver_port=0
--tserver_master_addrs=127.25.92.254:46225
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11462
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.92.195
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971406251336-25971-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:14:24.133392 11462 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:14:24.134575 11462 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260805 23:14:24.145301 11462 server_base.cc:1034] running on GCE node
W20260805 23:14:24.145236 11468 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:14:24.145242 11469 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:14:24.145249 11471 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:14:24.145987 11462 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:14:24.146723 11462 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:14:24.147912 11462 hybrid_clock.cc:648] HybridClock initialized: now 1785971664147867 us; error 62 us; skew 500 ppm
Aug 05 23:14:24 dist-test-slave-ss4n krb5kdc[11072](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971664, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:14:24.151074 11462 init.cc:377] Logged in from keytab as kudu/127.25.92.195@KRBTEST.COM (short username kudu)
I20260805 23:14:24.151997 11462 webserver.cc:460] Webserver started at http://127.25.92.195:38251/ using document root <none> and password file <none>
I20260805 23:14:24.152555 11462 fs_manager.cc:359] Metadata directory not provided
I20260805 23:14:24.152618 11462 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:14:24.152796 11462 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:14:24.155143 11462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971406251336-25971-0/minicluster-data/ts-2/data/instance:
uuid: "e295f5d920e947b69844f6460ce0469c"
format_stamp: "Formatted at 2026-08-05 23:14:24 on dist-test-slave-ss4n"
server_key: "3eee1441164053c76b9965a50ffa1d97"
server_key_iv: "66af8142a9f129024287f37ab864585a"
server_key_version: "encryptionkey@0"
I20260805 23:14:24.155723 11462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971406251336-25971-0/minicluster-data/ts-2/wal/instance:
uuid: "e295f5d920e947b69844f6460ce0469c"
format_stamp: "Formatted at 2026-08-05 23:14:24 on dist-test-slave-ss4n"
server_key: "3eee1441164053c76b9965a50ffa1d97"
server_key_iv: "66af8142a9f129024287f37ab864585a"
server_key_version: "encryptionkey@0"
I20260805 23:14:24.160069 11462 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260805 23:14:24.163214 11478 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:14:24.165307 11462 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260805 23:14:24.165429 11462 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971406251336-25971-0/minicluster-data/ts-2/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971406251336-25971-0/minicluster-data/ts-2/wal
uuid: "e295f5d920e947b69844f6460ce0469c"
format_stamp: "Formatted at 2026-08-05 23:14:24 on dist-test-slave-ss4n"
server_key: "3eee1441164053c76b9965a50ffa1d97"
server_key_iv: "66af8142a9f129024287f37ab864585a"
server_key_version: "encryptionkey@0"
I20260805 23:14:24.165645 11462 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971406251336-25971-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971406251336-25971-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971406251336-25971-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:14:24.185842 11462 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:14:24.189560 11462 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:14:24.189805 11462 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:14:24.190500 11462 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:14:24.191462 11462 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:14:24.191517 11462 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:14:24.191565 11462 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:14:24.191610 11462 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:14:24.203117 11462 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.195:35125
I20260805 23:14:24.203140 11597 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.195:35125 every 8 connection(s)
I20260805 23:14:24.204142 11462 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971406251336-25971-0/minicluster-data/ts-2/data/info.pb
I20260805 23:14:24.205744 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 11462
I20260805 23:14:24.205878 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971406251336-25971-0/minicluster-data/ts-2/wal/instance
I20260805 23:14:24.206171 25971 external_mini_cluster.cc:1428] Setting key 14c43e6b3c6a79ed41b34f8f25d037bd
Aug 05 23:14:24 dist-test-slave-ss4n krb5kdc[11072](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971664, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.195@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM
I20260805 23:14:24.221134 11159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:24.206398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.195:52673 (local address 127.25.92.254:46225)
0805 23:14:24.206588 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0805 23:14:24.206595 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0805 23:14:24.207599 (+ 1004us) server_negotiation.cc:407] Connection header received
0805 23:14:24.209034 (+ 1435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:14:24.209037 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:14:24.209084 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:14:24.209156 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:14:24.210326 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:24.210874 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:24.212209 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:24.212430 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:24.212517 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:24.215534 (+ 3017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:14:24.215537 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:14:24.215539 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:14:24.215569 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:14:24.217966 (+ 2397us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:14:24.218695 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:14:24.218700 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:14:24.218702 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:14:24.218841 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:14:24.219367 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:14:24.219372 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:14:24.219373 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:14:24.219746 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:14:24.219874 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0805 23:14:24.220793 (+ 919us) server_negotiation.cc:299] Negotiation successful
0805 23:14:24.220974 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260805 23:14:24.222357 11600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:24.206867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:46225 (local address 127.25.92.195:52673)
0805 23:14:24.207440 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:24.207497 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0805 23:14:24.208833 (+ 1336us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:14:24.209295 (+ 462us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:14:24.209301 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:14:24.209689 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:14:24.210184 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:14:24.210198 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:14:24.211020 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:14:24.211023 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:14:24.212072 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:14:24.212083 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:14:24.212579 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:14:24.212583 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:14:24.212782 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:24.213592 (+ 810us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:14:24.213611 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:14:24.215352 (+ 1741us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:14:24.218174 (+ 2822us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:14:24.218180 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:14:24.218196 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:14:24.218512 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:14:24.218979 (+ 467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:14:24.218983 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:14:24.218985 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:14:24.219213 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:14:24.219905 (+ 692us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:14:24.219913 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:14:24.220354 (+ 441us) client_negotiation.cc:769] Sending connection context
0805 23:14:24.220653 (+ 299us) client_negotiation.cc:240] Negotiation successful
0805 23:14:24.221146 (+ 493us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":170,"threads_started":1}
I20260805 23:14:24.223801 11598 heartbeater.cc:347] Connected to a master server at 127.25.92.254:46225
I20260805 23:14:24.224095 11598 heartbeater.cc:464] Registering TS with master...
I20260805 23:14:24.224656 11598 heartbeater.cc:511] Master 127.25.92.254:46225 requested a full tablet report, sending...
I20260805 23:14:24.225899 11118 ts_manager.cc:194] Registered new tserver with Master: e295f5d920e947b69844f6460ce0469c (127.25.92.195:35125)
I20260805 23:14:24.226549 11118 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.92.195@KRBTEST.COM'} at 127.25.92.195:52673
I20260805 23:14:24.233076 25971 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260805 23:14:24.243860 11159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:24.235311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53310 (local address 127.25.92.254:46225)
0805 23:14:24.235483 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0805 23:14:24.235488 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:14:24.235641 (+ 153us) server_negotiation.cc:407] Connection header received
0805 23:14:24.235877 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:14:24.235881 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:14:24.235931 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:14:24.236020 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:14:24.236764 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:24.237298 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:24.238249 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:24.238494 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:24.238580 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:24.240311 (+ 1731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:14:24.240314 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:14:24.240316 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:14:24.240341 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:14:24.242029 (+ 1688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:14:24.242552 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:14:24.242555 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:14:24.242556 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:14:24.242644 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:14:24.242954 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:14:24.242957 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:14:24.242959 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:14:24.243208 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:14:24.243324 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0805 23:14:24.243543 (+ 219us) server_negotiation.cc:299] Negotiation successful
0805 23:14:24.243646 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260805 23:14:24.247340 11118 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53310:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260805 23:14:24.250682 11118 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260805 23:14:24.266862 11609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:24.259737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.193:32955 (local address 127.0.0.1:56154)
0805 23:14:24.261033 (+ 1296us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:24.261049 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0805 23:14:24.261194 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:14:24.261945 (+ 751us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:14:24.261947 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:14:24.261962 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:14:24.262073 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:14:24.262080 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:14:24.263268 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:14:24.263273 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:14:24.264890 (+ 1617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:14:24.264901 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:14:24.266202 (+ 1301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:14:24.266208 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:14:24.266335 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:24.266376 (+ 41us) client_negotiation.cc:769] Sending connection context
0805 23:14:24.266524 (+ 148us) client_negotiation.cc:240] Negotiation successful
0805 23:14:24.266672 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1209,"mutex_wait_us":907,"thread_start_us":132,"threads_started":1}
I20260805 23:14:24.266901 11614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:24.260565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56154 (local address 127.25.92.193:32955)
0805 23:14:24.260883 (+ 318us) server_negotiation.cc:206] Beginning negotiation
0805 23:14:24.260890 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0805 23:14:24.261124 (+ 234us) server_negotiation.cc:407] Connection header received
0805 23:14:24.261351 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:14:24.261415 (+ 64us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:14:24.261648 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:14:24.261764 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:14:24.262227 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:24.263156 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:24.265043 (+ 1887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:24.266044 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:24.266179 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:24.266359 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0805 23:14:24.266584 (+ 225us) server_negotiation.cc:299] Negotiation successful
0805 23:14:24.266721 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":217,"thread_start_us":126,"threads_started":1}
I20260805 23:14:24.267503 11610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:24.259737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.194:46869 (local address 127.0.0.1:59848)
0805 23:14:24.260692 (+ 955us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:24.260753 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0805 23:14:24.260892 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:14:24.261580 (+ 688us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:14:24.261586 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:14:24.261608 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:14:24.261787 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:14:24.261795 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:14:24.263605 (+ 1810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:14:24.263608 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:14:24.264914 (+ 1306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:14:24.264920 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:14:24.266960 (+ 2040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:14:24.266966 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:14:24.267077 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:24.267111 (+ 34us) client_negotiation.cc:769] Sending connection context
0805 23:14:24.267240 (+ 129us) client_negotiation.cc:240] Negotiation successful
0805 23:14:24.267327 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":773,"mutex_wait_us":61,"thread_start_us":117,"threads_started":1}
I20260805 23:14:24.267550 11611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:24.259919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59848 (local address 127.25.92.194:46869)
0805 23:14:24.260479 (+ 560us) server_negotiation.cc:206] Beginning negotiation
0805 23:14:24.260495 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0805 23:14:24.261007 (+ 512us) server_negotiation.cc:407] Connection header received
0805 23:14:24.261109 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:14:24.261115 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:14:24.261352 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:14:24.261563 (+ 211us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:14:24.261934 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:24.263271 (+ 1337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:24.265649 (+ 2378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:24.266600 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:24.266730 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:24.266875 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0805 23:14:24.267276 (+ 401us) server_negotiation.cc:299] Negotiation successful
0805 23:14:24.267382 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":408,"thread_start_us":177,"threads_started":1}
I20260805 23:14:24.268395 11612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:24.260165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33098 (local address 127.25.92.195:35125)
0805 23:14:24.261201 (+ 1036us) server_negotiation.cc:206] Beginning negotiation
0805 23:14:24.261213 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0805 23:14:24.261456 (+ 243us) server_negotiation.cc:407] Connection header received
0805 23:14:24.261706 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:14:24.261713 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:14:24.261933 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:14:24.262112 (+ 179us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:14:24.262412 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:24.263763 (+ 1351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:24.265339 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:24.266100 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:24.266257 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:24.266409 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0805 23:14:24.268102 (+ 1693us) server_negotiation.cc:299] Negotiation successful
0805 23:14:24.268228 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":938,"thread_start_us":149,"threads_started":1}
I20260805 23:14:24.266873 11613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:24.259904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.195:35125 (local address 127.0.0.1:33098)
0805 23:14:24.261321 (+ 1417us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:24.261337 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0805 23:14:24.261514 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:14:24.262091 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:14:24.262094 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:14:24.262113 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:14:24.262236 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:14:24.262243 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:14:24.263937 (+ 1694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:14:24.263941 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:14:24.265101 (+ 1160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:14:24.265115 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:14:24.266483 (+ 1368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:14:24.266490 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:14:24.266605 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:24.266639 (+ 34us) client_negotiation.cc:769] Sending connection context
0805 23:14:24.266712 (+ 73us) client_negotiation.cc:240] Negotiation successful
0805 23:14:24.266774 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":984,"mutex_wait_us":5,"thread_start_us":119,"threads_started":1}
I20260805 23:14:24.271082 11390 tablet_service.cc:1467] Processing CreateTablet for tablet edd76d50af97405791c275657f819f95 (DEFAULT_TABLE table=test-table [id=5f56743a2c64484786c503c2248513c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:14:24.271081 11249 tablet_service.cc:1467] Processing CreateTablet for tablet edd76d50af97405791c275657f819f95 (DEFAULT_TABLE table=test-table [id=5f56743a2c64484786c503c2248513c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:14:24.271715 11527 tablet_service.cc:1467] Processing CreateTablet for tablet edd76d50af97405791c275657f819f95 (DEFAULT_TABLE table=test-table [id=5f56743a2c64484786c503c2248513c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:14:24.272374 11390 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet edd76d50af97405791c275657f819f95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:14:24.272374 11249 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet edd76d50af97405791c275657f819f95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:14:24.272841 11527 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet edd76d50af97405791c275657f819f95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:14:24.279814 11615 tablet_bootstrap.cc:492] T edd76d50af97405791c275657f819f95 P e295f5d920e947b69844f6460ce0469c: Bootstrap starting.
I20260805 23:14:24.281108 11616 tablet_bootstrap.cc:492] T edd76d50af97405791c275657f819f95 P 3520975d65424a2f90b8bd1d59ac27a3: Bootstrap starting.
I20260805 23:14:24.281608 11617 tablet_bootstrap.cc:492] T edd76d50af97405791c275657f819f95 P 28bf3bd95d17438e8a3669056f58c8fd: Bootstrap starting.
I20260805 23:14:24.282701 11615 tablet_bootstrap.cc:654] T edd76d50af97405791c275657f819f95 P e295f5d920e947b69844f6460ce0469c: Neither blocks nor log segments found. Creating new log.
I20260805 23:14:24.283821 11617 tablet_bootstrap.cc:654] T edd76d50af97405791c275657f819f95 P 28bf3bd95d17438e8a3669056f58c8fd: Neither blocks nor log segments found. Creating new log.
I20260805 23:14:24.284210 11615 log.cc:824] T edd76d50af97405791c275657f819f95 P e295f5d920e947b69844f6460ce0469c: Log is configured to *not* fsync() on all Append() calls
I20260805 23:14:24.285162 11617 log.cc:824] T edd76d50af97405791c275657f819f95 P 28bf3bd95d17438e8a3669056f58c8fd: Log is configured to *not* fsync() on all Append() calls
I20260805 23:14:24.287091 11617 tablet_bootstrap.cc:492] T edd76d50af97405791c275657f819f95 P 28bf3bd95d17438e8a3669056f58c8fd: No bootstrap required, opened a new log
I20260805 23:14:24.287163 11615 tablet_bootstrap.cc:492] T edd76d50af97405791c275657f819f95 P e295f5d920e947b69844f6460ce0469c: No bootstrap required, opened a new log
I20260805 23:14:24.287228 11616 tablet_bootstrap.cc:654] T edd76d50af97405791c275657f819f95 P 3520975d65424a2f90b8bd1d59ac27a3: Neither blocks nor log segments found. Creating new log.
I20260805 23:14:24.287335 11617 ts_tablet_manager.cc:1397] T edd76d50af97405791c275657f819f95 P 28bf3bd95d17438e8a3669056f58c8fd: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260805 23:14:24.287423 11615 ts_tablet_manager.cc:1397] T edd76d50af97405791c275657f819f95 P e295f5d920e947b69844f6460ce0469c: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260805 23:14:24.288190 11616 log.cc:824] T edd76d50af97405791c275657f819f95 P 3520975d65424a2f90b8bd1d59ac27a3: Log is configured to *not* fsync() on all Append() calls
I20260805 23:14:24.290803 11616 tablet_bootstrap.cc:492] T edd76d50af97405791c275657f819f95 P 3520975d65424a2f90b8bd1d59ac27a3: No bootstrap required, opened a new log
I20260805 23:14:24.291126 11616 ts_tablet_manager.cc:1397] T edd76d50af97405791c275657f819f95 P 3520975d65424a2f90b8bd1d59ac27a3: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260805 23:14:24.291438 11617 raft_consensus.cc:348] T edd76d50af97405791c275657f819f95 P 28bf3bd95d17438e8a3669056f58c8fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28bf3bd95d17438e8a3669056f58c8fd" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 46869 } } peers { permanent_uuid: "3520975d65424a2f90b8bd1d59ac27a3" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 32955 } } peers { permanent_uuid: "e295f5d920e947b69844f6460ce0469c" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 35125 } }
I20260805 23:14:24.291736 11617 raft_consensus.cc:374] T edd76d50af97405791c275657f819f95 P 28bf3bd95d17438e8a3669056f58c8fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:14:24.291819 11617 raft_consensus.cc:729] T edd76d50af97405791c275657f819f95 P 28bf3bd95d17438e8a3669056f58c8fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28bf3bd95d17438e8a3669056f58c8fd, State: Initialized, Role: FOLLOWER
I20260805 23:14:24.291816 11615 raft_consensus.cc:348] T edd76d50af97405791c275657f819f95 P e295f5d920e947b69844f6460ce0469c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28bf3bd95d17438e8a3669056f58c8fd" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 46869 } } peers { permanent_uuid: "3520975d65424a2f90b8bd1d59ac27a3" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 32955 } } peers { permanent_uuid: "e295f5d920e947b69844f6460ce0469c" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 35125 } }
I20260805 23:14:24.292210 11615 raft_consensus.cc:374] T edd76d50af97405791c275657f819f95 P e295f5d920e947b69844f6460ce0469c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:14:24.292296 11615 raft_consensus.cc:729] T edd76d50af97405791c275657f819f95 P e295f5d920e947b69844f6460ce0469c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e295f5d920e947b69844f6460ce0469c, State: Initialized, Role: FOLLOWER
I20260805 23:14:24.292347 11617 consensus_queue.cc:260] T edd76d50af97405791c275657f819f95 P 28bf3bd95d17438e8a3669056f58c8fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28bf3bd95d17438e8a3669056f58c8fd" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 46869 } } peers { permanent_uuid: "3520975d65424a2f90b8bd1d59ac27a3" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 32955 } } peers { permanent_uuid: "e295f5d920e947b69844f6460ce0469c" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 35125 } }
I20260805 23:14:24.292855 11615 consensus_queue.cc:260] T edd76d50af97405791c275657f819f95 P e295f5d920e947b69844f6460ce0469c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28bf3bd95d17438e8a3669056f58c8fd" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 46869 } } peers { permanent_uuid: "3520975d65424a2f90b8bd1d59ac27a3" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 32955 } } peers { permanent_uuid: "e295f5d920e947b69844f6460ce0469c" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 35125 } }
I20260805 23:14:24.293156 11457 heartbeater.cc:503] Master 127.25.92.254:46225 was elected leader, sending a full tablet report...
I20260805 23:14:24.293583 11617 ts_tablet_manager.cc:1428] T edd76d50af97405791c275657f819f95 P 28bf3bd95d17438e8a3669056f58c8fd: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
I20260805 23:14:24.294220 11615 ts_tablet_manager.cc:1428] T edd76d50af97405791c275657f819f95 P e295f5d920e947b69844f6460ce0469c: Time spent starting tablet: real 0.007s user 0.002s sys 0.004s
I20260805 23:14:24.294610 11598 heartbeater.cc:503] Master 127.25.92.254:46225 was elected leader, sending a full tablet report...
I20260805 23:14:24.297976 11616 raft_consensus.cc:348] T edd76d50af97405791c275657f819f95 P 3520975d65424a2f90b8bd1d59ac27a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28bf3bd95d17438e8a3669056f58c8fd" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 46869 } } peers { permanent_uuid: "3520975d65424a2f90b8bd1d59ac27a3" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 32955 } } peers { permanent_uuid: "e295f5d920e947b69844f6460ce0469c" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 35125 } }
I20260805 23:14:24.298607 11616 raft_consensus.cc:374] T edd76d50af97405791c275657f819f95 P 3520975d65424a2f90b8bd1d59ac27a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:14:24.298767 11616 raft_consensus.cc:729] T edd76d50af97405791c275657f819f95 P 3520975d65424a2f90b8bd1d59ac27a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3520975d65424a2f90b8bd1d59ac27a3, State: Initialized, Role: FOLLOWER
I20260805 23:14:24.299434 11616 consensus_queue.cc:260] T edd76d50af97405791c275657f819f95 P 3520975d65424a2f90b8bd1d59ac27a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28bf3bd95d17438e8a3669056f58c8fd" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 46869 } } peers { permanent_uuid: "3520975d65424a2f90b8bd1d59ac27a3" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 32955 } } peers { permanent_uuid: "e295f5d920e947b69844f6460ce0469c" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 35125 } }
I20260805 23:14:24.300346 11318 heartbeater.cc:503] Master 127.25.92.254:46225 was elected leader, sending a full tablet report...
I20260805 23:14:24.300750 11616 ts_tablet_manager.cc:1428] T edd76d50af97405791c275657f819f95 P 3520975d65424a2f90b8bd1d59ac27a3: Time spent starting tablet: real 0.010s user 0.007s sys 0.000s
I20260805 23:14:24.306172 11622 raft_consensus.cc:482] T edd76d50af97405791c275657f819f95 P e295f5d920e947b69844f6460ce0469c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 23:14:24.306357 11622 raft_consensus.cc:504] T edd76d50af97405791c275657f819f95 P e295f5d920e947b69844f6460ce0469c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28bf3bd95d17438e8a3669056f58c8fd" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 46869 } } peers { permanent_uuid: "3520975d65424a2f90b8bd1d59ac27a3" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 32955 } } peers { permanent_uuid: "e295f5d920e947b69844f6460ce0469c" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 35125 } }
I20260805 23:14:24.307690 11622 leader_election.cc:288] T edd76d50af97405791c275657f819f95 P e295f5d920e947b69844f6460ce0469c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 28bf3bd95d17438e8a3669056f58c8fd (127.25.92.194:46869), 3520975d65424a2f90b8bd1d59ac27a3 (127.25.92.193:32955)
I20260805 23:14:24.311793 11611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:24.307902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.195:58433 (local address 127.25.92.194:46869)
0805 23:14:24.308073 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0805 23:14:24.308080 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0805 23:14:24.308102 (+ 22us) server_negotiation.cc:407] Connection header received
0805 23:14:24.308292 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:14:24.308295 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:14:24.308384 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:14:24.308469 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:14:24.308817 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:24.309673 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:24.310799 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:24.311214 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:24.311349 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:24.311405 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0805 23:14:24.311593 (+ 188us) server_negotiation.cc:299] Negotiation successful
0805 23:14:24.311674 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260805 23:14:24.311793 11600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:24.307812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.194:46869 (local address 127.25.92.195:58433)
0805 23:14:24.308022 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:24.308040 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0805 23:14:24.308134 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:14:24.308503 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:14:24.308506 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:14:24.308526 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:14:24.308680 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:14:24.308689 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:14:24.309813 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:14:24.309818 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:14:24.310639 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:14:24.310650 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:14:24.311342 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:14:24.311345 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:14:24.311454 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:24.311483 (+ 29us) client_negotiation.cc:769] Sending connection context
0805 23:14:24.311574 (+ 91us) client_negotiation.cc:240] Negotiation successful
0805 23:14:24.311655 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124}
I20260805 23:14:24.312183 11624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:24.307949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.193:32955 (local address 127.25.92.195:43481)
0805 23:14:24.308297 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:24.308309 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0805 23:14:24.308412 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:14:24.308709 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:14:24.308714 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:14:24.308737 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:14:24.308875 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:14:24.308880 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:14:24.309858 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:14:24.309861 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:14:24.310888 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:14:24.310899 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:14:24.311660 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:14:24.311664 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:14:24.311779 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:24.311809 (+ 30us) client_negotiation.cc:769] Sending connection context
0805 23:14:24.311912 (+ 103us) client_negotiation.cc:240] Negotiation successful
0805 23:14:24.312030 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":162,"threads_started":1}
I20260805 23:14:24.312285 11614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:24.308184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.195:43481 (local address 127.25.92.193:32955)
0805 23:14:24.308303 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0805 23:14:24.308306 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 23:14:24.308392 (+ 86us) server_negotiation.cc:407] Connection header received
0805 23:14:24.308544 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:14:24.308547 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:14:24.308597 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:14:24.308673 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:14:24.308983 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:24.309731 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:24.311033 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:24.311506 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:24.311664 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:24.311717 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0805 23:14:24.312065 (+ 348us) server_negotiation.cc:299] Negotiation successful
0805 23:14:24.312150 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260805 23:14:24.312544 11411 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "edd76d50af97405791c275657f819f95" candidate_uuid: "e295f5d920e947b69844f6460ce0469c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bf3bd95d17438e8a3669056f58c8fd" is_pre_election: true
I20260805 23:14:24.312870 11411 raft_consensus.cc:2454] T edd76d50af97405791c275657f819f95 P 28bf3bd95d17438e8a3669056f58c8fd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e295f5d920e947b69844f6460ce0469c in term 0.
I20260805 23:14:24.312919 11269 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "edd76d50af97405791c275657f819f95" candidate_uuid: "e295f5d920e947b69844f6460ce0469c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3520975d65424a2f90b8bd1d59ac27a3" is_pre_election: true
I20260805 23:14:24.313263 11269 raft_consensus.cc:2454] T edd76d50af97405791c275657f819f95 P 3520975d65424a2f90b8bd1d59ac27a3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e295f5d920e947b69844f6460ce0469c in term 0.
I20260805 23:14:24.313455 11482 leader_election.cc:302] T edd76d50af97405791c275657f819f95 P e295f5d920e947b69844f6460ce0469c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bf3bd95d17438e8a3669056f58c8fd, e295f5d920e947b69844f6460ce0469c; no voters:
I20260805 23:14:24.313851 11622 raft_consensus.cc:2781] T edd76d50af97405791c275657f819f95 P e295f5d920e947b69844f6460ce0469c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260805 23:14:24.313953 11622 raft_consensus.cc:482] T edd76d50af97405791c275657f819f95 P e295f5d920e947b69844f6460ce0469c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260805 23:14:24.314013 11622 raft_consensus.cc:3037] T edd76d50af97405791c275657f819f95 P e295f5d920e947b69844f6460ce0469c [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:14:24.315177 11622 raft_consensus.cc:504] T edd76d50af97405791c275657f819f95 P e295f5d920e947b69844f6460ce0469c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28bf3bd95d17438e8a3669056f58c8fd" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 46869 } } peers { permanent_uuid: "3520975d65424a2f90b8bd1d59ac27a3" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 32955 } } peers { permanent_uuid: "e295f5d920e947b69844f6460ce0469c" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 35125 } }
I20260805 23:14:24.315622 11622 leader_election.cc:288] T edd76d50af97405791c275657f819f95 P e295f5d920e947b69844f6460ce0469c [CANDIDATE]: Term 1 election: Requested vote from peers 28bf3bd95d17438e8a3669056f58c8fd (127.25.92.194:46869), 3520975d65424a2f90b8bd1d59ac27a3 (127.25.92.193:32955)
I20260805 23:14:24.316004 11411 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "edd76d50af97405791c275657f819f95" candidate_uuid: "e295f5d920e947b69844f6460ce0469c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bf3bd95d17438e8a3669056f58c8fd"
I20260805 23:14:24.316049 11269 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "edd76d50af97405791c275657f819f95" candidate_uuid: "e295f5d920e947b69844f6460ce0469c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3520975d65424a2f90b8bd1d59ac27a3"
I20260805 23:14:24.316150 11411 raft_consensus.cc:3037] T edd76d50af97405791c275657f819f95 P 28bf3bd95d17438e8a3669056f58c8fd [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:14:24.316221 11269 raft_consensus.cc:3037] T edd76d50af97405791c275657f819f95 P 3520975d65424a2f90b8bd1d59ac27a3 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:14:24.317265 11411 raft_consensus.cc:2454] T edd76d50af97405791c275657f819f95 P 28bf3bd95d17438e8a3669056f58c8fd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e295f5d920e947b69844f6460ce0469c in term 1.
I20260805 23:14:24.317440 11269 raft_consensus.cc:2454] T edd76d50af97405791c275657f819f95 P 3520975d65424a2f90b8bd1d59ac27a3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e295f5d920e947b69844f6460ce0469c in term 1.
I20260805 23:14:24.317698 11482 leader_election.cc:302] T edd76d50af97405791c275657f819f95 P e295f5d920e947b69844f6460ce0469c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bf3bd95d17438e8a3669056f58c8fd, e295f5d920e947b69844f6460ce0469c; no voters:
I20260805 23:14:24.317922 11622 raft_consensus.cc:2781] T edd76d50af97405791c275657f819f95 P e295f5d920e947b69844f6460ce0469c [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:14:24.318198 11622 raft_consensus.cc:686] T edd76d50af97405791c275657f819f95 P e295f5d920e947b69844f6460ce0469c [term 1 LEADER]: Becoming Leader. State: Replica: e295f5d920e947b69844f6460ce0469c, State: Running, Role: LEADER
I20260805 23:14:24.318557 11622 consensus_queue.cc:237] T edd76d50af97405791c275657f819f95 P e295f5d920e947b69844f6460ce0469c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28bf3bd95d17438e8a3669056f58c8fd" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 46869 } } peers { permanent_uuid: "3520975d65424a2f90b8bd1d59ac27a3" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 32955 } } peers { permanent_uuid: "e295f5d920e947b69844f6460ce0469c" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 35125 } }
I20260805 23:14:24.322317 11117 catalog_manager.cc:5495] T edd76d50af97405791c275657f819f95 P e295f5d920e947b69844f6460ce0469c reported cstate change: term changed from 0 to 1, leader changed from <none> to e295f5d920e947b69844f6460ce0469c (127.25.92.195). New cstate: current_term: 1 leader_uuid: "e295f5d920e947b69844f6460ce0469c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28bf3bd95d17438e8a3669056f58c8fd" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 46869 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3520975d65424a2f90b8bd1d59ac27a3" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 32955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e295f5d920e947b69844f6460ce0469c" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 35125 } health_report { overall_health: HEALTHY } } }
I20260805 23:14:24.349620 11612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:24.345090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33102 (local address 127.25.92.195:35125)
0805 23:14:24.345250 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0805 23:14:24.345255 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:14:24.345272 (+ 17us) server_negotiation.cc:407] Connection header received
0805 23:14:24.345388 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:14:24.345391 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:14:24.345457 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:14:24.345694 (+ 237us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 23:14:24.346021 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:24.346538 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:24.347660 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:24.347910 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:24.348009 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:24.348445 (+ 436us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 23:14:24.348950 (+ 505us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 23:14:24.349067 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0805 23:14:24.349234 (+ 167us) server_negotiation.cc:299] Negotiation successful
0805 23:14:24.349358 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260805 23:14:24.356778 11269 raft_consensus.cc:1261] T edd76d50af97405791c275657f819f95 P 3520975d65424a2f90b8bd1d59ac27a3 [term 1 FOLLOWER]: Refusing update from remote peer e295f5d920e947b69844f6460ce0469c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:14:24.356778 11411 raft_consensus.cc:1261] T edd76d50af97405791c275657f819f95 P 28bf3bd95d17438e8a3669056f58c8fd [term 1 FOLLOWER]: Refusing update from remote peer e295f5d920e947b69844f6460ce0469c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:14:24.357930 11622 consensus_queue.cc:1035] T edd76d50af97405791c275657f819f95 P e295f5d920e947b69844f6460ce0469c [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bf3bd95d17438e8a3669056f58c8fd" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 46869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:14:24.358232 11625 consensus_queue.cc:1035] T edd76d50af97405791c275657f819f95 P e295f5d920e947b69844f6460ce0469c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3520975d65424a2f90b8bd1d59ac27a3" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 32955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:14:24.366120 11628 mvcc.cc:205] Tried to move back new op lower bound from 7315339937195552768 to 7315339937051844608. Current Snapshot: MvccSnapshot[applied={T|T < 7315339937195552768}]
I20260805 23:14:24.367456 11629 mvcc.cc:205] Tried to move back new op lower bound from 7315339937195552768 to 7315339937051844608. Current Snapshot: MvccSnapshot[applied={T|T < 7315339937195552768}]
I20260805 23:14:24.377365 11117 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53310:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:14:24.377632 11117 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53310:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:14:24.380607 11117 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 10c45b269fa448c4af9996257a81341e: Sending DeleteTablet for 3 replicas of tablet edd76d50af97405791c275657f819f95
I20260805 23:14:24.381580 11249 tablet_service.cc:1514] Processing DeleteTablet for tablet edd76d50af97405791c275657f819f95 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:14:24 UTC) from {username='kudu', principal='kudu/127.25.92.254@KRBTEST.COM'} at 127.0.0.1:56154
I20260805 23:14:24.381603 11390 tablet_service.cc:1514] Processing DeleteTablet for tablet edd76d50af97405791c275657f819f95 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:14:24 UTC) from {username='kudu', principal='kudu/127.25.92.254@KRBTEST.COM'} at 127.0.0.1:59848
I20260805 23:14:24.382652 11527 tablet_service.cc:1514] Processing DeleteTablet for tablet edd76d50af97405791c275657f819f95 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:14:24 UTC) from {username='kudu', principal='kudu/127.25.92.254@KRBTEST.COM'} at 127.0.0.1:33098
I20260805 23:14:24.382886 11637 tablet_replica.cc:330] T edd76d50af97405791c275657f819f95 P 3520975d65424a2f90b8bd1d59ac27a3: stopping tablet replica
I20260805 23:14:24.383241 11637 raft_consensus.cc:2229] T edd76d50af97405791c275657f819f95 P 3520975d65424a2f90b8bd1d59ac27a3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 23:14:24.383642 11637 raft_consensus.cc:2258] T edd76d50af97405791c275657f819f95 P 3520975d65424a2f90b8bd1d59ac27a3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:14:24.383849 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 11184
I20260805 23:14:24.388440 11639 tablet_replica.cc:330] T edd76d50af97405791c275657f819f95 P e295f5d920e947b69844f6460ce0469c: stopping tablet replica
I20260805 23:14:24.388952 11639 raft_consensus.cc:2229] T edd76d50af97405791c275657f819f95 P e295f5d920e947b69844f6460ce0469c [term 1 LEADER]: Raft consensus shutting down.
I20260805 23:14:24.389254 11638 tablet_replica.cc:330] T edd76d50af97405791c275657f819f95 P 28bf3bd95d17438e8a3669056f58c8fd: stopping tablet replica
I20260805 23:14:24.389461 11639 raft_consensus.cc:2258] T edd76d50af97405791c275657f819f95 P e295f5d920e947b69844f6460ce0469c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:14:24.389627 11638 raft_consensus.cc:2229] T edd76d50af97405791c275657f819f95 P 28bf3bd95d17438e8a3669056f58c8fd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 23:14:24.390007 11638 raft_consensus.cc:2258] T edd76d50af97405791c275657f819f95 P 28bf3bd95d17438e8a3669056f58c8fd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:14:24.391122 11639 ts_tablet_manager.cc:1905] T edd76d50af97405791c275657f819f95 P e295f5d920e947b69844f6460ce0469c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:14:24.394928 11638 ts_tablet_manager.cc:1905] T edd76d50af97405791c275657f819f95 P 28bf3bd95d17438e8a3669056f58c8fd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:14:24.395852 11639 ts_tablet_manager.cc:1918] T edd76d50af97405791c275657f819f95 P e295f5d920e947b69844f6460ce0469c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 23:14:24.395978 11639 log.cc:1196] T edd76d50af97405791c275657f819f95 P e295f5d920e947b69844f6460ce0469c: Deleting WAL directory at /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971406251336-25971-0/minicluster-data/ts-2/wal/wals/edd76d50af97405791c275657f819f95
I20260805 23:14:24.396377 11639 ts_tablet_manager.cc:1939] T edd76d50af97405791c275657f819f95 P e295f5d920e947b69844f6460ce0469c: Deleting consensus metadata
W20260805 23:14:24.397009 11104 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.25.92.193:32955 (error 108)
I20260805 23:14:24.397850 11103 catalog_manager.cc:4841] TS e295f5d920e947b69844f6460ce0469c (127.25.92.195:35125): tablet edd76d50af97405791c275657f819f95 (table test-table [id=5f56743a2c64484786c503c2248513c2]) successfully deleted
I20260805 23:14:24.398281 11638 ts_tablet_manager.cc:1918] T edd76d50af97405791c275657f819f95 P 28bf3bd95d17438e8a3669056f58c8fd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 23:14:24.398440 11638 log.cc:1196] T edd76d50af97405791c275657f819f95 P 28bf3bd95d17438e8a3669056f58c8fd: Deleting WAL directory at /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971406251336-25971-0/minicluster-data/ts-1/wal/wals/edd76d50af97405791c275657f819f95
I20260805 23:14:24.398453 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 11323
I20260805 23:14:24.398869 11613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:24.398454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.193:32955 (local address 127.0.0.1:56162)
0805 23:14:24.398675 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:24.398756 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.193:32955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
W20260805 23:14:24.400182 11104 catalog_manager.cc:4584] TS 3520975d65424a2f90b8bd1d59ac27a3 (127.25.92.193:32955): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet edd76d50af97405791c275657f819f95: Network error: Client connection negotiation failed: client connection to 127.25.92.193:32955: connect: Connection refused (error 111)
I20260805 23:14:24.405726 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 11462
I20260805 23:14:24.407542 11613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:24.405976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.194:46869 (local address 127.0.0.1:59854)
0805 23:14:24.407351 (+ 1375us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:24.407428 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.194:46869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1267}
I20260805 23:14:24.412398 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 11085
2026-08-05T23:14:24Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2509 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-05T23:14:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:14:24Z Disabled control of system clock
I20260805 23:14:24.448323 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971406251336-25971-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971406251336-25971-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971406251336-25971-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.254:44381
--webserver_interface=127.25.92.254
--webserver_port=0
--builtin_ntp_servers=127.25.92.212:37487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.92.254:44381
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260805 23:14:24.563814 11644 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:14:24.564134 11644 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:14:24.564175 11644 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:14:24.568475 11644 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:14:24.568563 11644 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:14:24.568579 11644 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:14:24.568596 11644 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:14:24.568612 11644 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:14:24.573835 11644 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:37487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971406251336-25971-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971406251336-25971-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.92.254:44381
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.254:44381
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.92.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11644
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971406251336-25971-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:14:24.575069 11644 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:14:24.576639 11644 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260805 23:14:24.586158 11644 server_base.cc:1034] running on GCE node
W20260805 23:14:24.586071 11652 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:14:24.586081 11650 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:14:24.586076 11649 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:14:24.587071 11644 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:14:24.588318 11644 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:14:24.589592 11644 hybrid_clock.cc:648] HybridClock initialized: now 1785971664589578 us; error 38 us; skew 500 ppm
I20260805 23:14:24.592032 11644 webserver.cc:460] Webserver started at http://127.25.92.254:44885/ using document root <none> and password file <none>
I20260805 23:14:24.592758 11644 fs_manager.cc:359] Metadata directory not provided
I20260805 23:14:24.592841 11644 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:14:24.593132 11644 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:14:24.595784 11644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971406251336-25971-0/minicluster-data/master-0/data/instance:
uuid: "4f7872ebf2f14292919eac0ae17fffc0"
format_stamp: "Formatted at 2026-08-05 23:14:24 on dist-test-slave-ss4n"
server_key: "fe8a296f43df4163183ee0ee00501d9c"
server_key_iv: "59fe9195db4237cb8f9c80a148aa6e20"
server_key_version: "encryptionkey@0"
I20260805 23:14:24.596525 11644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971406251336-25971-0/minicluster-data/master-0/wal/instance:
uuid: "4f7872ebf2f14292919eac0ae17fffc0"
format_stamp: "Formatted at 2026-08-05 23:14:24 on dist-test-slave-ss4n"
server_key: "fe8a296f43df4163183ee0ee00501d9c"
server_key_iv: "59fe9195db4237cb8f9c80a148aa6e20"
server_key_version: "encryptionkey@0"
I20260805 23:14:24.601274 11644 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260805 23:14:24.604772 11660 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:14:24.606319 11644 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:14:24.606509 11644 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971406251336-25971-0/minicluster-data/master-0/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971406251336-25971-0/minicluster-data/master-0/wal
uuid: "4f7872ebf2f14292919eac0ae17fffc0"
format_stamp: "Formatted at 2026-08-05 23:14:24 on dist-test-slave-ss4n"
server_key: "fe8a296f43df4163183ee0ee00501d9c"
server_key_iv: "59fe9195db4237cb8f9c80a148aa6e20"
server_key_version: "encryptionkey@0"
I20260805 23:14:24.606660 11644 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971406251336-25971-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971406251336-25971-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971406251336-25971-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:14:24.622498 11644 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:14:24.623409 11644 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:14:24.623616 11644 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:14:24.631592 11644 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.254:44381
I20260805 23:14:24.631636 11714 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.254:44381 every 8 connection(s)
I20260805 23:14:24.632680 11644 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
I20260805 23:14:24.635840 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 11644
I20260805 23:14:24.636063 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971406251336-25971-0/minicluster-data/master-0/wal/instance
I20260805 23:14:24.636744 11715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:14:24.636746 25971 external_mini_cluster.cc:1428] Setting key d4a0034569f56b493214cac42a7a37b6
I20260805 23:14:24.645078 11715 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f7872ebf2f14292919eac0ae17fffc0: Bootstrap starting.
I20260805 23:14:24.648411 11718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:24.638581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49458 (local address 127.25.92.254:44381)
0805 23:14:24.639411 (+ 830us) server_negotiation.cc:206] Beginning negotiation
0805 23:14:24.639428 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0805 23:14:24.639468 (+ 40us) server_negotiation.cc:407] Connection header received
0805 23:14:24.640418 (+ 950us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:14:24.640445 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:14:24.640890 (+ 445us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:14:24.641271 (+ 381us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:14:24.641816 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:24.642827 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:24.643786 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:24.644296 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:24.644492 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:24.645476 (+ 984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:14:24.645516 (+ 40us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:14:24.645534 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:14:24.646147 (+ 613us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:14:24.646233 (+ 86us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:14:24.646265 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:14:24.646504 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0805 23:14:24.646687 (+ 183us) server_negotiation.cc:299] Negotiation successful
0805 23:14:24.647368 (+ 681us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":447,"thread_start_us":215,"threads_started":1}
I20260805 23:14:24.648701 11715 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4f7872ebf2f14292919eac0ae17fffc0: Neither blocks nor log segments found. Creating new log.
I20260805 23:14:24.649729 11715 log.cc:824] T 00000000000000000000000000000000 P 4f7872ebf2f14292919eac0ae17fffc0: Log is configured to *not* fsync() on all Append() calls
I20260805 23:14:24.652101 11715 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f7872ebf2f14292919eac0ae17fffc0: No bootstrap required, opened a new log
I20260805 23:14:24.656123 11715 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4f7872ebf2f14292919eac0ae17fffc0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f7872ebf2f14292919eac0ae17fffc0" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 44381 } }
I20260805 23:14:24.656394 11715 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4f7872ebf2f14292919eac0ae17fffc0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:14:24.656457 11715 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4f7872ebf2f14292919eac0ae17fffc0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f7872ebf2f14292919eac0ae17fffc0, State: Initialized, Role: FOLLOWER
I20260805 23:14:24.656946 11715 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4f7872ebf2f14292919eac0ae17fffc0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f7872ebf2f14292919eac0ae17fffc0" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 44381 } }
I20260805 23:14:24.657094 11715 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4f7872ebf2f14292919eac0ae17fffc0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:14:24.657193 11715 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4f7872ebf2f14292919eac0ae17fffc0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:14:24.657316 11715 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4f7872ebf2f14292919eac0ae17fffc0 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:14:24.658589 11715 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4f7872ebf2f14292919eac0ae17fffc0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f7872ebf2f14292919eac0ae17fffc0" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 44381 } }
I20260805 23:14:24.658957 11715 leader_election.cc:302] T 00000000000000000000000000000000 P 4f7872ebf2f14292919eac0ae17fffc0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4f7872ebf2f14292919eac0ae17fffc0; no voters:
I20260805 23:14:24.659301 11715 leader_election.cc:288] T 00000000000000000000000000000000 P 4f7872ebf2f14292919eac0ae17fffc0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 23:14:24.659457 11720 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4f7872ebf2f14292919eac0ae17fffc0 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:14:24.659734 11720 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4f7872ebf2f14292919eac0ae17fffc0 [term 1 LEADER]: Becoming Leader. State: Replica: 4f7872ebf2f14292919eac0ae17fffc0, State: Running, Role: LEADER
I20260805 23:14:24.660050 11720 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4f7872ebf2f14292919eac0ae17fffc0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f7872ebf2f14292919eac0ae17fffc0" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 44381 } }
I20260805 23:14:24.660619 11715 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4f7872ebf2f14292919eac0ae17fffc0 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:14:24.661824 11722 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f7872ebf2f14292919eac0ae17fffc0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4f7872ebf2f14292919eac0ae17fffc0. Latest consensus state: current_term: 1 leader_uuid: "4f7872ebf2f14292919eac0ae17fffc0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f7872ebf2f14292919eac0ae17fffc0" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 44381 } } }
I20260805 23:14:24.661957 11722 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f7872ebf2f14292919eac0ae17fffc0 [sys.catalog]: This master's current role is: LEADER
I20260805 23:14:24.661927 11721 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f7872ebf2f14292919eac0ae17fffc0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4f7872ebf2f14292919eac0ae17fffc0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f7872ebf2f14292919eac0ae17fffc0" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 44381 } } }
I20260805 23:14:24.662025 11721 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f7872ebf2f14292919eac0ae17fffc0 [sys.catalog]: This master's current role is: LEADER
I20260805 23:14:24.662415 11729 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:14:24.665776 11729 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:14:24.673413 11729 catalog_manager.cc:1347] Generated new cluster ID: a6ecaaae0a1e458f9507b35b0ea75b1f
I20260805 23:14:24.673557 11729 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:14:24.698911 11729 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:14:24.700282 11729 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:14:24.709362 11729 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4f7872ebf2f14292919eac0ae17fffc0: Generated new TSK 0
I20260805 23:14:24.710335 11729 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260805 23:14:24.718951 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971406251336-25971-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971406251336-25971-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971406251336-25971-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971406251336-25971-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.92.193:0
--local_ip_for_outbound_sockets=127.25.92.193
--webserver_interface=127.25.92.193
--webserver_port=0
--tserver_master_addrs=127.25.92.254:44381
--builtin_ntp_servers=127.25.92.212:37487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260805 23:14:24.842905 11739 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:14:24.843230 11739 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:14:24.843272 11739 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:14:24.847672 11739 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:14:24.847752 11739 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:14:24.847813 11739 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.92.193
I20260805 23:14:24.853261 11739 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:37487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971406251336-25971-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971406251336-25971-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.92.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971406251336-25971-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.92.193
--webserver_port=0
--tserver_master_addrs=127.25.92.254:44381
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11739
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.92.193
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971406251336-25971-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:14:24.854571 11739 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:14:24.856171 11739 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:14:24.866847 11745 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:14:24.866870 11747 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:14:24.866847 11749 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:14:24.867192 11739 server_base.cc:1034] running on GCE node
I20260805 23:14:24.867792 11739 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:14:24.868523 11739 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:14:24.869725 11739 hybrid_clock.cc:648] HybridClock initialized: now 1785971664869693 us; error 56 us; skew 500 ppm
I20260805 23:14:24.871867 11739 webserver.cc:460] Webserver started at http://127.25.92.193:35937/ using document root <none> and password file <none>
I20260805 23:14:24.872502 11739 fs_manager.cc:359] Metadata directory not provided
I20260805 23:14:24.872602 11739 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:14:24.872857 11739 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:14:24.875213 11739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971406251336-25971-0/minicluster-data/ts-0/data/instance:
uuid: "7c0d34750d204ce28d84caaaf789e6f9"
format_stamp: "Formatted at 2026-08-05 23:14:24 on dist-test-slave-ss4n"
server_key: "ee07dcd1925af6c4ea4e2adf551ff9c2"
server_key_iv: "1d68d4cba7ed1f03e15ab35f2b6ddb26"
server_key_version: "encryptionkey@0"
I20260805 23:14:24.875949 11739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971406251336-25971-0/minicluster-data/ts-0/wal/instance:
uuid: "7c0d34750d204ce28d84caaaf789e6f9"
format_stamp: "Formatted at 2026-08-05 23:14:24 on dist-test-slave-ss4n"
server_key: "ee07dcd1925af6c4ea4e2adf551ff9c2"
server_key_iv: "1d68d4cba7ed1f03e15ab35f2b6ddb26"
server_key_version: "encryptionkey@0"
I20260805 23:14:24.879978 11739 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260805 23:14:24.882774 11755 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:14:24.884220 11739 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:14:24.884344 11739 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971406251336-25971-0/minicluster-data/ts-0/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971406251336-25971-0/minicluster-data/ts-0/wal
uuid: "7c0d34750d204ce28d84caaaf789e6f9"
format_stamp: "Formatted at 2026-08-05 23:14:24 on dist-test-slave-ss4n"
server_key: "ee07dcd1925af6c4ea4e2adf551ff9c2"
server_key_iv: "1d68d4cba7ed1f03e15ab35f2b6ddb26"
server_key_version: "encryptionkey@0"
I20260805 23:14:24.884526 11739 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971406251336-25971-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971406251336-25971-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971406251336-25971-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:14:24.896898 11739 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:14:24.897781 11739 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:14:24.897989 11739 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:14:24.898635 11739 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:14:24.899665 11739 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:14:24.899713 11739 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:14:24.899783 11739 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:14:24.899813 11739 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:14:24.910073 11739 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.193:36155
I20260805 23:14:24.910092 11870 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.193:36155 every 8 connection(s)
I20260805 23:14:24.911087 11739 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971406251336-25971-0/minicluster-data/ts-0/data/info.pb
I20260805 23:14:24.916867 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 11739
I20260805 23:14:24.916985 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971406251336-25971-0/minicluster-data/ts-0/wal/instance
I20260805 23:14:24.917275 25971 external_mini_cluster.cc:1428] Setting key c42df6fbb870dceec06400f57f35d3e8
I20260805 23:14:24.919673 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971406251336-25971-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971406251336-25971-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971406251336-25971-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971406251336-25971-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.92.194:0
--local_ip_for_outbound_sockets=127.25.92.194
--webserver_interface=127.25.92.194
--webserver_port=0
--tserver_master_addrs=127.25.92.254:44381
--builtin_ntp_servers=127.25.92.212:37487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260805 23:14:24.922607 11718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:24.913173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.193:51937 (local address 127.25.92.254:44381)
0805 23:14:24.913338 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0805 23:14:24.913343 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:14:24.914626 (+ 1283us) server_negotiation.cc:407] Connection header received
0805 23:14:24.916258 (+ 1632us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:14:24.916261 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:14:24.916326 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:14:24.916419 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:14:24.917695 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:24.918372 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:24.919871 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:24.920259 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:24.920390 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:24.921933 (+ 1543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:14:24.921939 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:14:24.921945 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:14:24.921962 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:14:24.921989 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:14:24.921998 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:14:24.922089 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0805 23:14:24.922281 (+ 192us) server_negotiation.cc:299] Negotiation successful
0805 23:14:24.922383 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260805 23:14:24.923612 11874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:24.913634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:44381 (local address 127.25.92.193:51937)
0805 23:14:24.914438 (+ 804us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:24.914496 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0805 23:14:24.916016 (+ 1520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:14:24.916631 (+ 615us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:14:24.916639 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:14:24.916740 (+ 101us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:14:24.917427 (+ 687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:14:24.917446 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:14:24.918527 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:14:24.918531 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:14:24.919693 (+ 1162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:14:24.919705 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:14:24.920621 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:14:24.920625 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:14:24.920816 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:24.921617 (+ 801us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0805 23:14:24.921619 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:14:24.921669 (+ 50us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0805 23:14:24.921751 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:14:24.922098 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:14:24.922107 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:14:24.922112 (+ 5us) client_negotiation.cc:769] Sending connection context
0805 23:14:24.922235 (+ 123us) client_negotiation.cc:240] Negotiation successful
0805 23:14:24.922589 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":163,"threads_started":1}
I20260805 23:14:24.925369 11872 heartbeater.cc:347] Connected to a master server at 127.25.92.254:44381
I20260805 23:14:24.925694 11872 heartbeater.cc:464] Registering TS with master...
I20260805 23:14:24.926303 11872 heartbeater.cc:511] Master 127.25.92.254:44381 requested a full tablet report, sending...
I20260805 23:14:24.928025 11678 ts_manager.cc:194] Registered new tserver with Master: 7c0d34750d204ce28d84caaaf789e6f9 (127.25.92.193:36155)
I20260805 23:14:24.929219 11678 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.25.92.193:51937
W20260805 23:14:25.039085 11875 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:14:25.039434 11875 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:14:25.039510 11875 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:14:25.043877 11875 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:14:25.043987 11875 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:14:25.044091 11875 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.92.194
I20260805 23:14:25.049147 11875 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:37487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971406251336-25971-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971406251336-25971-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.92.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971406251336-25971-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.92.194
--webserver_port=0
--tserver_master_addrs=127.25.92.254:44381
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11875
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.92.194
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971406251336-25971-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:14:25.050441 11875 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:14:25.051903 11875 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:14:25.062623 11881 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:14:25.062623 11882 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:14:25.062897 11875 server_base.cc:1034] running on GCE node
W20260805 23:14:25.063066 11884 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:14:25.063475 11875 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:14:25.064172 11875 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:14:25.065368 11875 hybrid_clock.cc:648] HybridClock initialized: now 1785971665065335 us; error 46 us; skew 500 ppm
I20260805 23:14:25.067881 11875 webserver.cc:460] Webserver started at http://127.25.92.194:35285/ using document root <none> and password file <none>
I20260805 23:14:25.068698 11875 fs_manager.cc:359] Metadata directory not provided
I20260805 23:14:25.068805 11875 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:14:25.069190 11875 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:14:25.071830 11875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971406251336-25971-0/minicluster-data/ts-1/data/instance:
uuid: "da88700f605c4ee9852cf03471ba1e05"
format_stamp: "Formatted at 2026-08-05 23:14:25 on dist-test-slave-ss4n"
server_key: "f3009289442a88db931c3af368ced655"
server_key_iv: "186a59d13096f2abe85b6bbf32caf172"
server_key_version: "encryptionkey@0"
I20260805 23:14:25.072451 11875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971406251336-25971-0/minicluster-data/ts-1/wal/instance:
uuid: "da88700f605c4ee9852cf03471ba1e05"
format_stamp: "Formatted at 2026-08-05 23:14:25 on dist-test-slave-ss4n"
server_key: "f3009289442a88db931c3af368ced655"
server_key_iv: "186a59d13096f2abe85b6bbf32caf172"
server_key_version: "encryptionkey@0"
I20260805 23:14:25.076607 11875 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260805 23:14:25.079432 11891 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:14:25.080901 11875 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260805 23:14:25.081080 11875 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971406251336-25971-0/minicluster-data/ts-1/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971406251336-25971-0/minicluster-data/ts-1/wal
uuid: "da88700f605c4ee9852cf03471ba1e05"
format_stamp: "Formatted at 2026-08-05 23:14:25 on dist-test-slave-ss4n"
server_key: "f3009289442a88db931c3af368ced655"
server_key_iv: "186a59d13096f2abe85b6bbf32caf172"
server_key_version: "encryptionkey@0"
I20260805 23:14:25.081234 11875 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971406251336-25971-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971406251336-25971-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971406251336-25971-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:14:25.101576 11875 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:14:25.102355 11875 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:14:25.102562 11875 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:14:25.103188 11875 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:14:25.104245 11875 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:14:25.104315 11875 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:14:25.104391 11875 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:14:25.104435 11875 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:14:25.113894 11875 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.194:36849
I20260805 23:14:25.113934 12006 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.194:36849 every 8 connection(s)
I20260805 23:14:25.114831 11875 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971406251336-25971-0/minicluster-data/ts-1/data/info.pb
I20260805 23:14:25.117806 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 11875
I20260805 23:14:25.117951 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971406251336-25971-0/minicluster-data/ts-1/wal/instance
I20260805 23:14:25.118306 25971 external_mini_cluster.cc:1428] Setting key d92ab8a36e00a2f1b93610d942e4fc7f
I20260805 23:14:25.120491 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971406251336-25971-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971406251336-25971-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971406251336-25971-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971406251336-25971-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.92.195:0
--local_ip_for_outbound_sockets=127.25.92.195
--webserver_interface=127.25.92.195
--webserver_port=0
--tserver_master_addrs=127.25.92.254:44381
--builtin_ntp_servers=127.25.92.212:37487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260805 23:14:25.125365 11718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:25.116679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.194:59257 (local address 127.25.92.254:44381)
0805 23:14:25.116823 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0805 23:14:25.116827 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:14:25.118145 (+ 1318us) server_negotiation.cc:407] Connection header received
0805 23:14:25.119901 (+ 1756us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:14:25.119904 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:14:25.119960 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:14:25.120054 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:14:25.121061 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:25.121656 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:25.122832 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:25.123067 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:25.123182 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:25.124785 (+ 1603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:14:25.124788 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:14:25.124791 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:14:25.124806 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:14:25.124827 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:14:25.124836 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:14:25.124923 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0805 23:14:25.125122 (+ 199us) server_negotiation.cc:299] Negotiation successful
0805 23:14:25.125201 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260805 23:14:25.126534 12009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:25.117092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:44381 (local address 127.25.92.194:59257)
0805 23:14:25.117939 (+ 847us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:25.118002 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0805 23:14:25.119638 (+ 1636us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:14:25.120243 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:14:25.120251 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:14:25.120342 (+ 91us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:14:25.120917 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:14:25.120932 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:14:25.121787 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:14:25.121790 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:14:25.122693 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:14:25.122703 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:14:25.123235 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:14:25.123240 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:14:25.123415 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:25.124299 (+ 884us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0805 23:14:25.124301 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:14:25.124351 (+ 50us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0805 23:14:25.124434 (+ 83us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:14:25.124940 (+ 506us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:14:25.124953 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:14:25.124960 (+ 7us) client_negotiation.cc:769] Sending connection context
0805 23:14:25.125110 (+ 150us) client_negotiation.cc:240] Negotiation successful
0805 23:14:25.125469 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":115,"threads_started":1}
I20260805 23:14:25.127924 12007 heartbeater.cc:347] Connected to a master server at 127.25.92.254:44381
I20260805 23:14:25.128211 12007 heartbeater.cc:464] Registering TS with master...
I20260805 23:14:25.128803 12007 heartbeater.cc:511] Master 127.25.92.254:44381 requested a full tablet report, sending...
I20260805 23:14:25.130095 11678 ts_manager.cc:194] Registered new tserver with Master: da88700f605c4ee9852cf03471ba1e05 (127.25.92.194:36849)
I20260805 23:14:25.131021 11678 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.25.92.194:59257
W20260805 23:14:25.238225 12010 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:14:25.238579 12010 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:14:25.238621 12010 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:14:25.242748 12010 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:14:25.242817 12010 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:14:25.242877 12010 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.92.195
I20260805 23:14:25.247906 12010 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:37487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971406251336-25971-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971406251336-25971-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.92.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971406251336-25971-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.92.195
--webserver_port=0
--tserver_master_addrs=127.25.92.254:44381
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12010
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.92.195
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971406251336-25971-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:14:25.249027 12010 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:14:25.250420 12010 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:14:25.260478 12017 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:14:25.260581 12010 server_base.cc:1034] running on GCE node
W20260805 23:14:25.260462 12016 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:14:25.260462 12019 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:14:25.261193 12010 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:14:25.261994 12010 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:14:25.263192 12010 hybrid_clock.cc:648] HybridClock initialized: now 1785971665263184 us; error 46 us; skew 500 ppm
I20260805 23:14:25.265131 12010 webserver.cc:460] Webserver started at http://127.25.92.195:35805/ using document root <none> and password file <none>
I20260805 23:14:25.265838 12010 fs_manager.cc:359] Metadata directory not provided
I20260805 23:14:25.265938 12010 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:14:25.266199 12010 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:14:25.268497 12010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971406251336-25971-0/minicluster-data/ts-2/data/instance:
uuid: "78384740cb2a46839a0eab151ace8269"
format_stamp: "Formatted at 2026-08-05 23:14:25 on dist-test-slave-ss4n"
server_key: "5f4eb13691d681dbc4cb193905475240"
server_key_iv: "52fd7065475b8e88b9a754c4dce01c50"
server_key_version: "encryptionkey@0"
I20260805 23:14:25.269094 12010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971406251336-25971-0/minicluster-data/ts-2/wal/instance:
uuid: "78384740cb2a46839a0eab151ace8269"
format_stamp: "Formatted at 2026-08-05 23:14:25 on dist-test-slave-ss4n"
server_key: "5f4eb13691d681dbc4cb193905475240"
server_key_iv: "52fd7065475b8e88b9a754c4dce01c50"
server_key_version: "encryptionkey@0"
I20260805 23:14:25.272985 12010 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.003s
I20260805 23:14:25.275559 12025 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:14:25.276968 12010 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260805 23:14:25.277139 12010 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971406251336-25971-0/minicluster-data/ts-2/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971406251336-25971-0/minicluster-data/ts-2/wal
uuid: "78384740cb2a46839a0eab151ace8269"
format_stamp: "Formatted at 2026-08-05 23:14:25 on dist-test-slave-ss4n"
server_key: "5f4eb13691d681dbc4cb193905475240"
server_key_iv: "52fd7065475b8e88b9a754c4dce01c50"
server_key_version: "encryptionkey@0"
I20260805 23:14:25.277271 12010 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971406251336-25971-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971406251336-25971-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971406251336-25971-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:14:25.305646 12010 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:14:25.306411 12010 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:14:25.306629 12010 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:14:25.307291 12010 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:14:25.308295 12010 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:14:25.308378 12010 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:14:25.308451 12010 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:14:25.308506 12010 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:14:25.317965 12010 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.195:45805
I20260805 23:14:25.318079 12141 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.195:45805 every 8 connection(s)
I20260805 23:14:25.319056 12010 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971406251336-25971-0/minicluster-data/ts-2/data/info.pb
I20260805 23:14:25.328424 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 12010
I20260805 23:14:25.328578 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971406251336-25971-0/minicluster-data/ts-2/wal/instance
I20260805 23:14:25.328975 25971 external_mini_cluster.cc:1428] Setting key 75649b1cbbfcabf1eee133132f6d786a
I20260805 23:14:25.329241 11718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:25.321048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.195:39913 (local address 127.25.92.254:44381)
0805 23:14:25.321228 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0805 23:14:25.321232 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:14:25.322332 (+ 1100us) server_negotiation.cc:407] Connection header received
0805 23:14:25.323794 (+ 1462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:14:25.323798 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:14:25.323849 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:14:25.323958 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:14:25.324931 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:25.325466 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:25.326781 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:25.327034 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:25.327123 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:25.328489 (+ 1366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:14:25.328495 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:14:25.328498 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:14:25.328520 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:14:25.328549 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:14:25.328562 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:14:25.328655 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0805 23:14:25.328988 (+ 333us) server_negotiation.cc:299] Negotiation successful
0805 23:14:25.329070 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260805 23:14:25.330515 12144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:25.321444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:44381 (local address 127.25.92.195:39913)
0805 23:14:25.322163 (+ 719us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:25.322219 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0805 23:14:25.323556 (+ 1337us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:14:25.324134 (+ 578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:14:25.324141 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:14:25.324227 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:14:25.324783 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:14:25.324797 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:14:25.325704 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:14:25.325707 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:14:25.326652 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:14:25.326662 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:14:25.327244 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:14:25.327247 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:14:25.327414 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:25.328170 (+ 756us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0805 23:14:25.328172 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:14:25.328216 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0805 23:14:25.328292 (+ 76us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:14:25.328788 (+ 496us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:14:25.328799 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:14:25.328806 (+ 7us) client_negotiation.cc:769] Sending connection context
0805 23:14:25.328958 (+ 152us) client_negotiation.cc:240] Negotiation successful
0805 23:14:25.329311 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":116,"threads_started":1}
I20260805 23:14:25.331924 12142 heartbeater.cc:347] Connected to a master server at 127.25.92.254:44381
I20260805 23:14:25.332136 12142 heartbeater.cc:464] Registering TS with master...
I20260805 23:14:25.332628 12142 heartbeater.cc:511] Master 127.25.92.254:44381 requested a full tablet report, sending...
I20260805 23:14:25.333669 11678 ts_manager.cc:194] Registered new tserver with Master: 78384740cb2a46839a0eab151ace8269 (127.25.92.195:45805)
I20260805 23:14:25.334250 11678 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.25.92.195:39913
I20260805 23:14:25.343513 25971 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260805 23:14:25.349884 11718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:25.345818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49464 (local address 127.25.92.254:44381)
0805 23:14:25.345992 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0805 23:14:25.345996 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:14:25.346073 (+ 77us) server_negotiation.cc:407] Connection header received
0805 23:14:25.346209 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:14:25.346212 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:14:25.346256 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:14:25.346326 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:14:25.346616 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:25.347154 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:25.348103 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:25.348332 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:25.348414 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:25.349243 (+ 829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:14:25.349246 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:14:25.349248 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:14:25.349261 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:14:25.349279 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:14:25.349287 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:14:25.349345 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0805 23:14:25.349600 (+ 255us) server_negotiation.cc:299] Negotiation successful
0805 23:14:25.349688 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260805 23:14:25.353137 11678 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49464:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260805 23:14:25.356320 11678 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260805 23:14:25.374348 12156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:25.367132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.193:36155 (local address 127.0.0.1:33646)
0805 23:14:25.368377 (+ 1245us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:25.368420 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0805 23:14:25.368617 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:14:25.369279 (+ 662us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:14:25.369287 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:14:25.369329 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:14:25.369639 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:14:25.369646 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:14:25.371349 (+ 1703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:14:25.371354 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:14:25.372529 (+ 1175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:14:25.372540 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:14:25.373749 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:14:25.373756 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:14:25.373870 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:25.373909 (+ 39us) client_negotiation.cc:769] Sending connection context
0805 23:14:25.374001 (+ 92us) client_negotiation.cc:240] Negotiation successful
0805 23:14:25.374148 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1077,"mutex_wait_us":303,"spinlock_wait_cycles":22144,"thread_start_us":118,"threads_started":1}
I20260805 23:14:25.374729 12155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:25.366626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34258 (local address 127.25.92.195:45805)
0805 23:14:25.367586 (+ 960us) server_negotiation.cc:206] Beginning negotiation
0805 23:14:25.367603 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0805 23:14:25.368532 (+ 929us) server_negotiation.cc:407] Connection header received
0805 23:14:25.368822 (+ 290us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:14:25.368829 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:14:25.369064 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:14:25.369228 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:14:25.369801 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:25.371206 (+ 1405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:25.372912 (+ 1706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:25.373711 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:25.373808 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:25.374027 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0805 23:14:25.374425 (+ 398us) server_negotiation.cc:299] Negotiation successful
0805 23:14:25.374547 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":818,"thread_start_us":142,"threads_started":1}
I20260805 23:14:25.374836 12157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:25.367373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.194:36849 (local address 127.0.0.1:53622)
0805 23:14:25.368380 (+ 1007us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:25.368435 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0805 23:14:25.368608 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:14:25.369965 (+ 1357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:14:25.369968 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:14:25.369987 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:14:25.370096 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:14:25.370102 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:14:25.371200 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:14:25.371206 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:14:25.372770 (+ 1564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:14:25.372782 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:14:25.374395 (+ 1613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:14:25.374398 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:14:25.374486 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:25.374520 (+ 34us) client_negotiation.cc:769] Sending connection context
0805 23:14:25.374586 (+ 66us) client_negotiation.cc:240] Negotiation successful
0805 23:14:25.374665 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":912,"mutex_wait_us":79,"spinlock_wait_cycles":4480,"thread_start_us":116,"threads_started":1}
I20260805 23:14:25.375183 12159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:25.367839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53622 (local address 127.25.92.194:36849)
0805 23:14:25.368277 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0805 23:14:25.368289 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0805 23:14:25.368609 (+ 320us) server_negotiation.cc:407] Connection header received
0805 23:14:25.369433 (+ 824us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:14:25.369437 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:14:25.369656 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:14:25.369807 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:14:25.370231 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:25.371076 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:25.373595 (+ 2519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:25.374263 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:25.374394 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:25.374709 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0805 23:14:25.374759 (+ 50us) server_negotiation.cc:299] Negotiation successful
0805 23:14:25.375024 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":170,"threads_started":1}
I20260805 23:14:25.375568 12158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:25.367330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33646 (local address 127.25.92.193:36155)
0805 23:14:25.367828 (+ 498us) server_negotiation.cc:206] Beginning negotiation
0805 23:14:25.367843 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0805 23:14:25.368530 (+ 687us) server_negotiation.cc:407] Connection header received
0805 23:14:25.368835 (+ 305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:14:25.368847 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:14:25.369057 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:14:25.369228 (+ 171us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:14:25.369799 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:25.371202 (+ 1403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:25.372701 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:25.373589 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:25.375072 (+ 1483us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:25.375237 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0805 23:14:25.375307 (+ 70us) server_negotiation.cc:299] Negotiation successful
0805 23:14:25.375411 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":186,"threads_started":1}
I20260805 23:14:25.375813 12154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:25.366466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.195:45805 (local address 127.0.0.1:34258)
0805 23:14:25.368380 (+ 1914us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:25.368424 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0805 23:14:25.368608 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:14:25.369279 (+ 671us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:14:25.369287 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:14:25.369329 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:14:25.369621 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:14:25.369632 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:14:25.371322 (+ 1690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:14:25.371325 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:14:25.372774 (+ 1449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:14:25.372782 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:14:25.374088 (+ 1306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:14:25.374091 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:14:25.374170 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:25.374189 (+ 19us) client_negotiation.cc:769] Sending connection context
0805 23:14:25.374259 (+ 70us) client_negotiation.cc:240] Negotiation successful
0805 23:14:25.374314 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1675,"mutex_wait_us":202,"spinlock_wait_cycles":8448,"thread_start_us":123,"threads_started":1}
I20260805 23:14:25.378562 11803 tablet_service.cc:1467] Processing CreateTablet for tablet 5049a52ba55941a3ba4278aca85cfbca (DEFAULT_TABLE table=test-table [id=04618874711c4d07beafda145dd43dfa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:14:25.378564 12076 tablet_service.cc:1467] Processing CreateTablet for tablet 5049a52ba55941a3ba4278aca85cfbca (DEFAULT_TABLE table=test-table [id=04618874711c4d07beafda145dd43dfa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:14:25.378562 11940 tablet_service.cc:1467] Processing CreateTablet for tablet 5049a52ba55941a3ba4278aca85cfbca (DEFAULT_TABLE table=test-table [id=04618874711c4d07beafda145dd43dfa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:14:25.379648 11803 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5049a52ba55941a3ba4278aca85cfbca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:14:25.379648 11940 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5049a52ba55941a3ba4278aca85cfbca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:14:25.379648 12076 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5049a52ba55941a3ba4278aca85cfbca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:14:25.387189 12160 tablet_bootstrap.cc:492] T 5049a52ba55941a3ba4278aca85cfbca P 7c0d34750d204ce28d84caaaf789e6f9: Bootstrap starting.
I20260805 23:14:25.388486 12161 tablet_bootstrap.cc:492] T 5049a52ba55941a3ba4278aca85cfbca P da88700f605c4ee9852cf03471ba1e05: Bootstrap starting.
I20260805 23:14:25.389225 12162 tablet_bootstrap.cc:492] T 5049a52ba55941a3ba4278aca85cfbca P 78384740cb2a46839a0eab151ace8269: Bootstrap starting.
I20260805 23:14:25.390693 12160 tablet_bootstrap.cc:654] T 5049a52ba55941a3ba4278aca85cfbca P 7c0d34750d204ce28d84caaaf789e6f9: Neither blocks nor log segments found. Creating new log.
I20260805 23:14:25.391083 12161 tablet_bootstrap.cc:654] T 5049a52ba55941a3ba4278aca85cfbca P da88700f605c4ee9852cf03471ba1e05: Neither blocks nor log segments found. Creating new log.
I20260805 23:14:25.391605 12162 tablet_bootstrap.cc:654] T 5049a52ba55941a3ba4278aca85cfbca P 78384740cb2a46839a0eab151ace8269: Neither blocks nor log segments found. Creating new log.
I20260805 23:14:25.391733 12160 log.cc:824] T 5049a52ba55941a3ba4278aca85cfbca P 7c0d34750d204ce28d84caaaf789e6f9: Log is configured to *not* fsync() on all Append() calls
I20260805 23:14:25.391811 12161 log.cc:824] T 5049a52ba55941a3ba4278aca85cfbca P da88700f605c4ee9852cf03471ba1e05: Log is configured to *not* fsync() on all Append() calls
I20260805 23:14:25.392498 12162 log.cc:824] T 5049a52ba55941a3ba4278aca85cfbca P 78384740cb2a46839a0eab151ace8269: Log is configured to *not* fsync() on all Append() calls
I20260805 23:14:25.393893 12161 tablet_bootstrap.cc:492] T 5049a52ba55941a3ba4278aca85cfbca P da88700f605c4ee9852cf03471ba1e05: No bootstrap required, opened a new log
I20260805 23:14:25.394115 12161 ts_tablet_manager.cc:1397] T 5049a52ba55941a3ba4278aca85cfbca P da88700f605c4ee9852cf03471ba1e05: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260805 23:14:25.394145 12160 tablet_bootstrap.cc:492] T 5049a52ba55941a3ba4278aca85cfbca P 7c0d34750d204ce28d84caaaf789e6f9: No bootstrap required, opened a new log
I20260805 23:14:25.394347 12160 ts_tablet_manager.cc:1397] T 5049a52ba55941a3ba4278aca85cfbca P 7c0d34750d204ce28d84caaaf789e6f9: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260805 23:14:25.394675 12162 tablet_bootstrap.cc:492] T 5049a52ba55941a3ba4278aca85cfbca P 78384740cb2a46839a0eab151ace8269: No bootstrap required, opened a new log
I20260805 23:14:25.394912 12162 ts_tablet_manager.cc:1397] T 5049a52ba55941a3ba4278aca85cfbca P 78384740cb2a46839a0eab151ace8269: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260805 23:14:25.397945 12161 raft_consensus.cc:348] T 5049a52ba55941a3ba4278aca85cfbca P da88700f605c4ee9852cf03471ba1e05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78384740cb2a46839a0eab151ace8269" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 45805 } } peers { permanent_uuid: "da88700f605c4ee9852cf03471ba1e05" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 36849 } } peers { permanent_uuid: "7c0d34750d204ce28d84caaaf789e6f9" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 36155 } }
I20260805 23:14:25.398231 12161 raft_consensus.cc:374] T 5049a52ba55941a3ba4278aca85cfbca P da88700f605c4ee9852cf03471ba1e05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:14:25.398285 12161 raft_consensus.cc:729] T 5049a52ba55941a3ba4278aca85cfbca P da88700f605c4ee9852cf03471ba1e05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da88700f605c4ee9852cf03471ba1e05, State: Initialized, Role: FOLLOWER
I20260805 23:14:25.398746 12161 consensus_queue.cc:260] T 5049a52ba55941a3ba4278aca85cfbca P da88700f605c4ee9852cf03471ba1e05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78384740cb2a46839a0eab151ace8269" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 45805 } } peers { permanent_uuid: "da88700f605c4ee9852cf03471ba1e05" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 36849 } } peers { permanent_uuid: "7c0d34750d204ce28d84caaaf789e6f9" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 36155 } }
I20260805 23:14:25.399461 12007 heartbeater.cc:503] Master 127.25.92.254:44381 was elected leader, sending a full tablet report...
I20260805 23:14:25.399420 12160 raft_consensus.cc:348] T 5049a52ba55941a3ba4278aca85cfbca P 7c0d34750d204ce28d84caaaf789e6f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78384740cb2a46839a0eab151ace8269" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 45805 } } peers { permanent_uuid: "da88700f605c4ee9852cf03471ba1e05" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 36849 } } peers { permanent_uuid: "7c0d34750d204ce28d84caaaf789e6f9" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 36155 } }
I20260805 23:14:25.399772 12160 raft_consensus.cc:374] T 5049a52ba55941a3ba4278aca85cfbca P 7c0d34750d204ce28d84caaaf789e6f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:14:25.399842 12160 raft_consensus.cc:729] T 5049a52ba55941a3ba4278aca85cfbca P 7c0d34750d204ce28d84caaaf789e6f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c0d34750d204ce28d84caaaf789e6f9, State: Initialized, Role: FOLLOWER
I20260805 23:14:25.399681 12162 raft_consensus.cc:348] T 5049a52ba55941a3ba4278aca85cfbca P 78384740cb2a46839a0eab151ace8269 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78384740cb2a46839a0eab151ace8269" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 45805 } } peers { permanent_uuid: "da88700f605c4ee9852cf03471ba1e05" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 36849 } } peers { permanent_uuid: "7c0d34750d204ce28d84caaaf789e6f9" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 36155 } }
I20260805 23:14:25.399886 12161 ts_tablet_manager.cc:1428] T 5049a52ba55941a3ba4278aca85cfbca P da88700f605c4ee9852cf03471ba1e05: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260805 23:14:25.399945 12162 raft_consensus.cc:374] T 5049a52ba55941a3ba4278aca85cfbca P 78384740cb2a46839a0eab151ace8269 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:14:25.399999 12162 raft_consensus.cc:729] T 5049a52ba55941a3ba4278aca85cfbca P 78384740cb2a46839a0eab151ace8269 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78384740cb2a46839a0eab151ace8269, State: Initialized, Role: FOLLOWER
I20260805 23:14:25.400439 12162 consensus_queue.cc:260] T 5049a52ba55941a3ba4278aca85cfbca P 78384740cb2a46839a0eab151ace8269 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78384740cb2a46839a0eab151ace8269" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 45805 } } peers { permanent_uuid: "da88700f605c4ee9852cf03471ba1e05" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 36849 } } peers { permanent_uuid: "7c0d34750d204ce28d84caaaf789e6f9" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 36155 } }
I20260805 23:14:25.400429 12160 consensus_queue.cc:260] T 5049a52ba55941a3ba4278aca85cfbca P 7c0d34750d204ce28d84caaaf789e6f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78384740cb2a46839a0eab151ace8269" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 45805 } } peers { permanent_uuid: "da88700f605c4ee9852cf03471ba1e05" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 36849 } } peers { permanent_uuid: "7c0d34750d204ce28d84caaaf789e6f9" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 36155 } }
I20260805 23:14:25.401260 11872 heartbeater.cc:503] Master 127.25.92.254:44381 was elected leader, sending a full tablet report...
I20260805 23:14:25.401551 12142 heartbeater.cc:503] Master 127.25.92.254:44381 was elected leader, sending a full tablet report...
I20260805 23:14:25.401664 12160 ts_tablet_manager.cc:1428] T 5049a52ba55941a3ba4278aca85cfbca P 7c0d34750d204ce28d84caaaf789e6f9: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260805 23:14:25.402348 12162 ts_tablet_manager.cc:1428] T 5049a52ba55941a3ba4278aca85cfbca P 78384740cb2a46839a0eab151ace8269: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
W20260805 23:14:25.412331 11873 tablet.cc:2369] T 5049a52ba55941a3ba4278aca85cfbca P 7c0d34750d204ce28d84caaaf789e6f9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260805 23:14:25.570122 12143 tablet.cc:2369] T 5049a52ba55941a3ba4278aca85cfbca P 78384740cb2a46839a0eab151ace8269: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260805 23:14:25.616046 12008 tablet.cc:2369] T 5049a52ba55941a3ba4278aca85cfbca P da88700f605c4ee9852cf03471ba1e05: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260805 23:14:25.755671 12167 raft_consensus.cc:482] T 5049a52ba55941a3ba4278aca85cfbca P 78384740cb2a46839a0eab151ace8269 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 23:14:25.755949 12167 raft_consensus.cc:504] T 5049a52ba55941a3ba4278aca85cfbca P 78384740cb2a46839a0eab151ace8269 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78384740cb2a46839a0eab151ace8269" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 45805 } } peers { permanent_uuid: "da88700f605c4ee9852cf03471ba1e05" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 36849 } } peers { permanent_uuid: "7c0d34750d204ce28d84caaaf789e6f9" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 36155 } }
I20260805 23:14:25.756390 12166 raft_consensus.cc:482] T 5049a52ba55941a3ba4278aca85cfbca P da88700f605c4ee9852cf03471ba1e05 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 23:14:25.756620 12166 raft_consensus.cc:504] T 5049a52ba55941a3ba4278aca85cfbca P da88700f605c4ee9852cf03471ba1e05 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78384740cb2a46839a0eab151ace8269" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 45805 } } peers { permanent_uuid: "da88700f605c4ee9852cf03471ba1e05" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 36849 } } peers { permanent_uuid: "7c0d34750d204ce28d84caaaf789e6f9" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 36155 } }
I20260805 23:14:25.757345 12167 leader_election.cc:288] T 5049a52ba55941a3ba4278aca85cfbca P 78384740cb2a46839a0eab151ace8269 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers da88700f605c4ee9852cf03471ba1e05 (127.25.92.194:36849), 7c0d34750d204ce28d84caaaf789e6f9 (127.25.92.193:36155)
I20260805 23:14:25.757730 12166 leader_election.cc:288] T 5049a52ba55941a3ba4278aca85cfbca P da88700f605c4ee9852cf03471ba1e05 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 78384740cb2a46839a0eab151ace8269 (127.25.92.195:45805), 7c0d34750d204ce28d84caaaf789e6f9 (127.25.92.193:36155)
I20260805 23:14:25.762706 12144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:25.757631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.194:36849 (local address 127.25.92.195:56095)
0805 23:14:25.757851 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:25.757868 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0805 23:14:25.757976 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:14:25.758291 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:14:25.758294 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:14:25.758311 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:14:25.758441 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:14:25.758450 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:14:25.759813 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:14:25.759818 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:14:25.761204 (+ 1386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:14:25.761214 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:14:25.762235 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:14:25.762239 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:14:25.762337 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:25.762358 (+ 21us) client_negotiation.cc:769] Sending connection context
0805 23:14:25.762444 (+ 86us) client_negotiation.cc:240] Negotiation successful
0805 23:14:25.762525 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"mutex_wait_us":36}
I20260805 23:14:25.763075 12155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:25.758604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.194:57991 (local address 127.25.92.195:45805)
0805 23:14:25.758714 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0805 23:14:25.758720 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:14:25.758734 (+ 14us) server_negotiation.cc:407] Connection header received
0805 23:14:25.758876 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:14:25.758879 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:14:25.758928 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:14:25.758999 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:14:25.759367 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:25.760062 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:25.761442 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:25.762027 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:25.762124 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:25.762172 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0805 23:14:25.762567 (+ 395us) server_negotiation.cc:299] Negotiation successful
0805 23:14:25.762643 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":50}
I20260805 23:14:25.763494 12159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:25.757923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.195:56095 (local address 127.25.92.194:36849)
0805 23:14:25.758047 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0805 23:14:25.758052 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:14:25.758068 (+ 16us) server_negotiation.cc:407] Connection header received
0805 23:14:25.758121 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:14:25.758124 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:14:25.758169 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:14:25.758239 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:14:25.758903 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:25.759686 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:25.761354 (+ 1668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:25.761950 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:25.762067 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:25.762115 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0805 23:14:25.763271 (+ 1156us) server_negotiation.cc:299] Negotiation successful
0805 23:14:25.763357 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260805 23:14:25.763502 12158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:25.757796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.195:53379 (local address 127.25.92.193:36155)
0805 23:14:25.757971 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0805 23:14:25.757977 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:14:25.758222 (+ 245us) server_negotiation.cc:407] Connection header received
0805 23:14:25.758398 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:14:25.758404 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:14:25.758460 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:14:25.758513 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:14:25.759008 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:25.759750 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:25.762098 (+ 2348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:25.762761 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:25.762865 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:25.762913 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0805 23:14:25.763298 (+ 385us) server_negotiation.cc:299] Negotiation successful
0805 23:14:25.763378 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260805 23:14:25.763383 12170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:25.758281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.195:45805 (local address 127.25.92.194:57991)
0805 23:14:25.758664 (+ 383us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:25.758679 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0805 23:14:25.758765 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:14:25.759092 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:14:25.759095 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:14:25.759115 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:14:25.759239 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:14:25.759245 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:14:25.760164 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:14:25.760167 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:14:25.761297 (+ 1130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:14:25.761309 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:14:25.762317 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:14:25.762321 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:14:25.762431 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:25.762452 (+ 21us) client_negotiation.cc:769] Sending connection context
0805 23:14:25.762732 (+ 280us) client_negotiation.cc:240] Negotiation successful
0805 23:14:25.763046 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"mutex_wait_us":3,"thread_start_us":104,"threads_started":1}
I20260805 23:14:25.763934 12169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:25.757679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.193:36155 (local address 127.25.92.195:53379)
0805 23:14:25.758131 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:25.758147 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0805 23:14:25.758251 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:14:25.758678 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:14:25.758682 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:14:25.758699 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:14:25.758867 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:14:25.758874 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:14:25.759889 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:14:25.759892 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:14:25.761950 (+ 2058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:14:25.761961 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:14:25.763040 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:14:25.763043 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:14:25.763144 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:25.763178 (+ 34us) client_negotiation.cc:769] Sending connection context
0805 23:14:25.763691 (+ 513us) client_negotiation.cc:240] Negotiation successful
0805 23:14:25.763766 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":372,"mutex_wait_us":1,"thread_start_us":121,"threads_started":1}
I20260805 23:14:25.764027 12171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:25.758758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.194:49915 (local address 127.25.92.193:36155)
0805 23:14:25.759263 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0805 23:14:25.759267 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:14:25.759502 (+ 235us) server_negotiation.cc:407] Connection header received
0805 23:14:25.759719 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:14:25.759725 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:14:25.759780 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:14:25.759836 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:14:25.760270 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:25.761063 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:25.762595 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:25.763281 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:25.763370 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:25.763424 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0805 23:14:25.763823 (+ 399us) server_negotiation.cc:299] Negotiation successful
0805 23:14:25.763902 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":423,"thread_start_us":100,"threads_started":1}
I20260805 23:14:25.764111 11960 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5049a52ba55941a3ba4278aca85cfbca" candidate_uuid: "78384740cb2a46839a0eab151ace8269" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da88700f605c4ee9852cf03471ba1e05" is_pre_election: true
I20260805 23:14:25.764374 12172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:25.758585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.193:36155 (local address 127.25.92.194:49915)
0805 23:14:25.759428 (+ 843us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:25.759441 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0805 23:14:25.759575 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:14:25.759962 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:14:25.759964 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:14:25.759980 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:14:25.760102 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:14:25.760108 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:14:25.761215 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:14:25.761219 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:14:25.762465 (+ 1246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:14:25.762474 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:14:25.763509 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:14:25.763512 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:14:25.763676 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:25.763702 (+ 26us) client_negotiation.cc:769] Sending connection context
0805 23:14:25.764160 (+ 458us) client_negotiation.cc:240] Negotiation successful
0805 23:14:25.764235 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":785,"mutex_wait_us":358,"thread_start_us":56,"threads_started":1}
I20260805 23:14:25.764384 11960 raft_consensus.cc:2454] T 5049a52ba55941a3ba4278aca85cfbca P da88700f605c4ee9852cf03471ba1e05 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 78384740cb2a46839a0eab151ace8269 in term 0.
I20260805 23:14:25.764617 11823 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5049a52ba55941a3ba4278aca85cfbca" candidate_uuid: "78384740cb2a46839a0eab151ace8269" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0d34750d204ce28d84caaaf789e6f9" is_pre_election: true
I20260805 23:14:25.764760 12096 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5049a52ba55941a3ba4278aca85cfbca" candidate_uuid: "da88700f605c4ee9852cf03471ba1e05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78384740cb2a46839a0eab151ace8269" is_pre_election: true
I20260805 23:14:25.764926 12096 raft_consensus.cc:2454] T 5049a52ba55941a3ba4278aca85cfbca P 78384740cb2a46839a0eab151ace8269 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da88700f605c4ee9852cf03471ba1e05 in term 0.
I20260805 23:14:25.764928 11823 raft_consensus.cc:2454] T 5049a52ba55941a3ba4278aca85cfbca P 7c0d34750d204ce28d84caaaf789e6f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 78384740cb2a46839a0eab151ace8269 in term 0.
I20260805 23:14:25.765125 12028 leader_election.cc:302] T 5049a52ba55941a3ba4278aca85cfbca P 78384740cb2a46839a0eab151ace8269 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 78384740cb2a46839a0eab151ace8269, da88700f605c4ee9852cf03471ba1e05; no voters:
I20260805 23:14:25.765208 11822 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5049a52ba55941a3ba4278aca85cfbca" candidate_uuid: "da88700f605c4ee9852cf03471ba1e05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0d34750d204ce28d84caaaf789e6f9" is_pre_election: true
I20260805 23:14:25.765322 11822 raft_consensus.cc:2454] T 5049a52ba55941a3ba4278aca85cfbca P 7c0d34750d204ce28d84caaaf789e6f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da88700f605c4ee9852cf03471ba1e05 in term 0.
I20260805 23:14:25.765362 11894 leader_election.cc:302] T 5049a52ba55941a3ba4278aca85cfbca P da88700f605c4ee9852cf03471ba1e05 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 78384740cb2a46839a0eab151ace8269, da88700f605c4ee9852cf03471ba1e05; no voters:
I20260805 23:14:25.765452 12167 raft_consensus.cc:2781] T 5049a52ba55941a3ba4278aca85cfbca P 78384740cb2a46839a0eab151ace8269 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260805 23:14:25.765565 12167 raft_consensus.cc:482] T 5049a52ba55941a3ba4278aca85cfbca P 78384740cb2a46839a0eab151ace8269 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260805 23:14:25.765618 12167 raft_consensus.cc:3037] T 5049a52ba55941a3ba4278aca85cfbca P 78384740cb2a46839a0eab151ace8269 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:14:25.765677 12166 raft_consensus.cc:2781] T 5049a52ba55941a3ba4278aca85cfbca P da88700f605c4ee9852cf03471ba1e05 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260805 23:14:25.765776 12166 raft_consensus.cc:482] T 5049a52ba55941a3ba4278aca85cfbca P da88700f605c4ee9852cf03471ba1e05 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260805 23:14:25.765843 12166 raft_consensus.cc:3037] T 5049a52ba55941a3ba4278aca85cfbca P da88700f605c4ee9852cf03471ba1e05 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:14:25.766752 12167 raft_consensus.cc:504] T 5049a52ba55941a3ba4278aca85cfbca P 78384740cb2a46839a0eab151ace8269 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78384740cb2a46839a0eab151ace8269" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 45805 } } peers { permanent_uuid: "da88700f605c4ee9852cf03471ba1e05" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 36849 } } peers { permanent_uuid: "7c0d34750d204ce28d84caaaf789e6f9" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 36155 } }
I20260805 23:14:25.766999 12166 raft_consensus.cc:504] T 5049a52ba55941a3ba4278aca85cfbca P da88700f605c4ee9852cf03471ba1e05 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78384740cb2a46839a0eab151ace8269" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 45805 } } peers { permanent_uuid: "da88700f605c4ee9852cf03471ba1e05" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 36849 } } peers { permanent_uuid: "7c0d34750d204ce28d84caaaf789e6f9" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 36155 } }
I20260805 23:14:25.767169 12167 leader_election.cc:288] T 5049a52ba55941a3ba4278aca85cfbca P 78384740cb2a46839a0eab151ace8269 [CANDIDATE]: Term 1 election: Requested vote from peers da88700f605c4ee9852cf03471ba1e05 (127.25.92.194:36849), 7c0d34750d204ce28d84caaaf789e6f9 (127.25.92.193:36155)
I20260805 23:14:25.767407 12166 leader_election.cc:288] T 5049a52ba55941a3ba4278aca85cfbca P da88700f605c4ee9852cf03471ba1e05 [CANDIDATE]: Term 1 election: Requested vote from peers 78384740cb2a46839a0eab151ace8269 (127.25.92.195:45805), 7c0d34750d204ce28d84caaaf789e6f9 (127.25.92.193:36155)
I20260805 23:14:25.767606 11822 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5049a52ba55941a3ba4278aca85cfbca" candidate_uuid: "78384740cb2a46839a0eab151ace8269" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0d34750d204ce28d84caaaf789e6f9"
I20260805 23:14:25.767863 11822 raft_consensus.cc:3037] T 5049a52ba55941a3ba4278aca85cfbca P 7c0d34750d204ce28d84caaaf789e6f9 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:14:25.767944 11960 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5049a52ba55941a3ba4278aca85cfbca" candidate_uuid: "78384740cb2a46839a0eab151ace8269" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da88700f605c4ee9852cf03471ba1e05"
I20260805 23:14:25.767957 12096 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5049a52ba55941a3ba4278aca85cfbca" candidate_uuid: "da88700f605c4ee9852cf03471ba1e05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78384740cb2a46839a0eab151ace8269"
I20260805 23:14:25.768080 11823 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5049a52ba55941a3ba4278aca85cfbca" candidate_uuid: "da88700f605c4ee9852cf03471ba1e05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0d34750d204ce28d84caaaf789e6f9"
I20260805 23:14:25.768123 11960 raft_consensus.cc:2379] T 5049a52ba55941a3ba4278aca85cfbca P da88700f605c4ee9852cf03471ba1e05 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 78384740cb2a46839a0eab151ace8269 in current term 1: Already voted for candidate da88700f605c4ee9852cf03471ba1e05 in this term.
I20260805 23:14:25.768157 12096 raft_consensus.cc:2379] T 5049a52ba55941a3ba4278aca85cfbca P 78384740cb2a46839a0eab151ace8269 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate da88700f605c4ee9852cf03471ba1e05 in current term 1: Already voted for candidate 78384740cb2a46839a0eab151ace8269 in this term.
I20260805 23:14:25.768730 11895 leader_election.cc:302] T 5049a52ba55941a3ba4278aca85cfbca P da88700f605c4ee9852cf03471ba1e05 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da88700f605c4ee9852cf03471ba1e05; no voters: 78384740cb2a46839a0eab151ace8269, 7c0d34750d204ce28d84caaaf789e6f9
I20260805 23:14:25.768919 12166 raft_consensus.cc:2726] T 5049a52ba55941a3ba4278aca85cfbca P da88700f605c4ee9852cf03471ba1e05 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260805 23:14:25.769368 11822 raft_consensus.cc:2454] T 5049a52ba55941a3ba4278aca85cfbca P 7c0d34750d204ce28d84caaaf789e6f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 78384740cb2a46839a0eab151ace8269 in term 1.
I20260805 23:14:25.769747 12029 leader_election.cc:302] T 5049a52ba55941a3ba4278aca85cfbca P 78384740cb2a46839a0eab151ace8269 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 78384740cb2a46839a0eab151ace8269, 7c0d34750d204ce28d84caaaf789e6f9; no voters: da88700f605c4ee9852cf03471ba1e05
I20260805 23:14:25.769953 12167 raft_consensus.cc:2781] T 5049a52ba55941a3ba4278aca85cfbca P 78384740cb2a46839a0eab151ace8269 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:14:25.770223 12167 raft_consensus.cc:686] T 5049a52ba55941a3ba4278aca85cfbca P 78384740cb2a46839a0eab151ace8269 [term 1 LEADER]: Becoming Leader. State: Replica: 78384740cb2a46839a0eab151ace8269, State: Running, Role: LEADER
I20260805 23:14:25.770589 12167 consensus_queue.cc:237] T 5049a52ba55941a3ba4278aca85cfbca P 78384740cb2a46839a0eab151ace8269 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78384740cb2a46839a0eab151ace8269" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 45805 } } peers { permanent_uuid: "da88700f605c4ee9852cf03471ba1e05" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 36849 } } peers { permanent_uuid: "7c0d34750d204ce28d84caaaf789e6f9" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 36155 } }
I20260805 23:14:25.774103 11676 catalog_manager.cc:5495] T 5049a52ba55941a3ba4278aca85cfbca P 78384740cb2a46839a0eab151ace8269 reported cstate change: term changed from 0 to 1, leader changed from <none> to 78384740cb2a46839a0eab151ace8269 (127.25.92.195). New cstate: current_term: 1 leader_uuid: "78384740cb2a46839a0eab151ace8269" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78384740cb2a46839a0eab151ace8269" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 45805 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da88700f605c4ee9852cf03471ba1e05" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 36849 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c0d34750d204ce28d84caaaf789e6f9" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 36155 } health_report { overall_health: UNKNOWN } } }
I20260805 23:14:25.828706 12155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:25.824835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34260 (local address 127.25.92.195:45805)
0805 23:14:25.824996 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0805 23:14:25.825025 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0805 23:14:25.825048 (+ 23us) server_negotiation.cc:407] Connection header received
0805 23:14:25.825152 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:14:25.825155 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:14:25.825246 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:14:25.825367 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 23:14:25.825651 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:25.826195 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:25.827085 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:25.827327 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:25.827445 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:25.827781 (+ 336us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 23:14:25.828256 (+ 475us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 23:14:25.828403 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0805 23:14:25.828469 (+ 66us) server_negotiation.cc:299] Negotiation successful
0805 23:14:25.828536 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260805 23:14:25.835182 11960 raft_consensus.cc:1261] T 5049a52ba55941a3ba4278aca85cfbca P da88700f605c4ee9852cf03471ba1e05 [term 1 FOLLOWER]: Refusing update from remote peer 78384740cb2a46839a0eab151ace8269: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:14:25.835173 11822 raft_consensus.cc:1261] T 5049a52ba55941a3ba4278aca85cfbca P 7c0d34750d204ce28d84caaaf789e6f9 [term 1 FOLLOWER]: Refusing update from remote peer 78384740cb2a46839a0eab151ace8269: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:14:25.835878 12167 consensus_queue.cc:1035] T 5049a52ba55941a3ba4278aca85cfbca P 78384740cb2a46839a0eab151ace8269 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c0d34750d204ce28d84caaaf789e6f9" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 36155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:14:25.836122 12173 consensus_queue.cc:1035] T 5049a52ba55941a3ba4278aca85cfbca P 78384740cb2a46839a0eab151ace8269 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da88700f605c4ee9852cf03471ba1e05" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 36849 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:14:25.844503 12177 mvcc.cc:205] Tried to move back new op lower bound from 7315339943252721664 to 7315339942999289856. Current Snapshot: MvccSnapshot[applied={T|T < 7315339943252721664}]
I20260805 23:14:25.854432 11677 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49464:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:14:25.854667 11677 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49464:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:14:25.857406 11677 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4f7872ebf2f14292919eac0ae17fffc0: Sending DeleteTablet for 3 replicas of tablet 5049a52ba55941a3ba4278aca85cfbca
I20260805 23:14:25.858477 11803 tablet_service.cc:1514] Processing DeleteTablet for tablet 5049a52ba55941a3ba4278aca85cfbca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:14:25 UTC) from {username='slave'} at 127.0.0.1:33646
I20260805 23:14:25.858475 11940 tablet_service.cc:1514] Processing DeleteTablet for tablet 5049a52ba55941a3ba4278aca85cfbca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:14:25 UTC) from {username='slave'} at 127.0.0.1:53622
I20260805 23:14:25.858672 12076 tablet_service.cc:1514] Processing DeleteTablet for tablet 5049a52ba55941a3ba4278aca85cfbca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:14:25 UTC) from {username='slave'} at 127.0.0.1:34258
I20260805 23:14:25.859086 12189 tablet_replica.cc:330] T 5049a52ba55941a3ba4278aca85cfbca P 7c0d34750d204ce28d84caaaf789e6f9: stopping tablet replica
I20260805 23:14:25.859452 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 11739
I20260805 23:14:25.863234 12190 tablet_replica.cc:330] T 5049a52ba55941a3ba4278aca85cfbca P 78384740cb2a46839a0eab151ace8269: stopping tablet replica
I20260805 23:14:25.863597 12190 raft_consensus.cc:2229] T 5049a52ba55941a3ba4278aca85cfbca P 78384740cb2a46839a0eab151ace8269 [term 1 LEADER]: Raft consensus shutting down.
I20260805 23:14:25.863993 12190 raft_consensus.cc:2258] T 5049a52ba55941a3ba4278aca85cfbca P 78384740cb2a46839a0eab151ace8269 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:14:25.864923 12188 tablet_replica.cc:330] T 5049a52ba55941a3ba4278aca85cfbca P da88700f605c4ee9852cf03471ba1e05: stopping tablet replica
I20260805 23:14:25.865372 12188 raft_consensus.cc:2229] T 5049a52ba55941a3ba4278aca85cfbca P da88700f605c4ee9852cf03471ba1e05 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 23:14:25.865851 12188 raft_consensus.cc:2258] T 5049a52ba55941a3ba4278aca85cfbca P da88700f605c4ee9852cf03471ba1e05 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:14:25.867677 12190 ts_tablet_manager.cc:1905] T 5049a52ba55941a3ba4278aca85cfbca P 78384740cb2a46839a0eab151ace8269: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260805 23:14:25.867865 11664 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.25.92.193:36155 (error 108)
I20260805 23:14:25.868122 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 11875
I20260805 23:14:25.870739 12190 ts_tablet_manager.cc:1918] T 5049a52ba55941a3ba4278aca85cfbca P 78384740cb2a46839a0eab151ace8269: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 23:14:25.870863 12190 log.cc:1196] T 5049a52ba55941a3ba4278aca85cfbca P 78384740cb2a46839a0eab151ace8269: Deleting WAL directory at /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971406251336-25971-0/minicluster-data/ts-2/wal/wals/5049a52ba55941a3ba4278aca85cfbca
I20260805 23:14:25.871191 12190 ts_tablet_manager.cc:1939] T 5049a52ba55941a3ba4278aca85cfbca P 78384740cb2a46839a0eab151ace8269: Deleting consensus metadata
I20260805 23:14:25.871632 12154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:25.871326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.193:36155 (local address 127.0.0.1:33662)
0805 23:14:25.871466 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:25.871551 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.193:36155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260805 23:14:25.871874 11664 catalog_manager.cc:4584] TS 7c0d34750d204ce28d84caaaf789e6f9 (127.25.92.193:36155): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5049a52ba55941a3ba4278aca85cfbca: Network error: Client connection negotiation failed: client connection to 127.25.92.193:36155: connect: Connection refused (error 111)
I20260805 23:14:25.872515 11663 catalog_manager.cc:4841] TS 78384740cb2a46839a0eab151ace8269 (127.25.92.195:45805): tablet 5049a52ba55941a3ba4278aca85cfbca (table test-table [id=04618874711c4d07beafda145dd43dfa]) successfully deleted
I20260805 23:14:25.878253 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 12010
I20260805 23:14:25.879763 12154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:25.878515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.194:36849 (local address 127.0.0.1:53632)
0805 23:14:25.879566 (+ 1051us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:25.879633 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.194:36849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":944}
I20260805 23:14:25.885212 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 11644
2026-08-05T23:14:25Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1469 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971406251336-25971-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 23:14:25 dist-test-slave-ss4n krb5kdc[12194](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 23:14:25 dist-test-slave-ss4n krb5kdc[12194](info): set up 2 sockets
Aug 05 23:14:25 dist-test-slave-ss4n krb5kdc[12194](info): commencing operation
krb5kdc: starting...
W20260805 23:14:27.023834 25971 mini_kdc.cc:121] Time spent starting KDC: real 1.116s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971406251336-25971-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971406251336-25971-0/krb5kdc/test-user.keytab.
Aug 05 23:14:27 dist-test-slave-ss4n krb5kdc[12194](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971667, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T23:14:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:14:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.92.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.92.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.92.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab.
Entry for principal kudu/127.25.92.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab.
I20260805 23:14:27.172470 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971406251336-25971-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971406251336-25971-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971406251336-25971-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.254:39905
--webserver_interface=127.25.92.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab
--principal=kudu/127.25.92.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:33677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.92.254:39905
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:14:27.286700 12208 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:14:27.287039 12208 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:14:27.287114 12208 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:14:27.291225 12208 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:14:27.291323 12208 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:14:27.291360 12208 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:14:27.291416 12208 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:14:27.291452 12208 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:14:27.297071 12208 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:33677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971406251336-25971-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971406251336-25971-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.92.254:39905
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab
--principal=kudu/127.25.92.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.254:39905
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.92.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12208
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971406251336-25971-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:14:27.298266 12208 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:14:27.299341 12208 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:14:27.308385 12214 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:14:27.308509 12213 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:14:27.308491 12208 server_base.cc:1034] running on GCE node
W20260805 23:14:27.308429 12216 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:14:27.309254 12208 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:14:27.310509 12208 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:14:27.311759 12208 hybrid_clock.cc:648] HybridClock initialized: now 1785971667311711 us; error 63 us; skew 500 ppm
Aug 05 23:14:27 dist-test-slave-ss4n krb5kdc[12194](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971667, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:14:27.314899 12208 init.cc:377] Logged in from keytab as kudu/127.25.92.254@KRBTEST.COM (short username kudu)
I20260805 23:14:27.315958 12208 webserver.cc:460] Webserver started at http://127.25.92.254:36997/ using document root <none> and password file <none>
I20260805 23:14:27.316555 12208 fs_manager.cc:359] Metadata directory not provided
I20260805 23:14:27.316658 12208 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:14:27.316905 12208 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:14:27.319171 12208 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971406251336-25971-0/minicluster-data/master-0/data/instance:
uuid: "fae04452d9f048998e94a0ea113588b8"
format_stamp: "Formatted at 2026-08-05 23:14:27 on dist-test-slave-ss4n"
server_key: "2230c4d1118f6c83e8736a4149c20600"
server_key_iv: "3917f2bae5f7fc95a820ce7743114c89"
server_key_version: "encryptionkey@0"
I20260805 23:14:27.319787 12208 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971406251336-25971-0/minicluster-data/master-0/wal/instance:
uuid: "fae04452d9f048998e94a0ea113588b8"
format_stamp: "Formatted at 2026-08-05 23:14:27 on dist-test-slave-ss4n"
server_key: "2230c4d1118f6c83e8736a4149c20600"
server_key_iv: "3917f2bae5f7fc95a820ce7743114c89"
server_key_version: "encryptionkey@0"
I20260805 23:14:27.324004 12208 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260805 23:14:27.326898 12224 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:14:27.328377 12208 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260805 23:14:27.328576 12208 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971406251336-25971-0/minicluster-data/master-0/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971406251336-25971-0/minicluster-data/master-0/wal
uuid: "fae04452d9f048998e94a0ea113588b8"
format_stamp: "Formatted at 2026-08-05 23:14:27 on dist-test-slave-ss4n"
server_key: "2230c4d1118f6c83e8736a4149c20600"
server_key_iv: "3917f2bae5f7fc95a820ce7743114c89"
server_key_version: "encryptionkey@0"
I20260805 23:14:27.328735 12208 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971406251336-25971-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971406251336-25971-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971406251336-25971-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:14:27.353875 12208 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:14:27.357385 12208 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:14:27.357663 12208 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:14:27.365671 12208 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.254:39905
I20260805 23:14:27.365700 12277 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.254:39905 every 8 connection(s)
I20260805 23:14:27.366741 12208 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
I20260805 23:14:27.369704 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 12208
I20260805 23:14:27.369801 12278 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:14:27.369911 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971406251336-25971-0/minicluster-data/master-0/wal/instance
I20260805 23:14:27.370257 25971 external_mini_cluster.cc:1428] Setting key 081aeefb3ba546a9c259406b63e82c2a
I20260805 23:14:27.376579 12278 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fae04452d9f048998e94a0ea113588b8: Bootstrap starting.
I20260805 23:14:27.379285 12278 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fae04452d9f048998e94a0ea113588b8: Neither blocks nor log segments found. Creating new log.
Aug 05 23:14:27 dist-test-slave-ss4n krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971667, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM
I20260805 23:14:27.380280 12278 log.cc:824] T 00000000000000000000000000000000 P fae04452d9f048998e94a0ea113588b8: Log is configured to *not* fsync() on all Append() calls
I20260805 23:14:27.382709 12278 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fae04452d9f048998e94a0ea113588b8: No bootstrap required, opened a new log
I20260805 23:14:27.385610 12281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:27.371787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34218 (local address 127.25.92.254:39905)
0805 23:14:27.372288 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0805 23:14:27.372303 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0805 23:14:27.372338 (+ 35us) server_negotiation.cc:407] Connection header received
0805 23:14:27.373196 (+ 858us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:14:27.373216 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:14:27.373627 (+ 411us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:14:27.374017 (+ 390us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:14:27.374875 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:27.375875 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:27.376769 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:27.377322 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:27.377536 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:27.380483 (+ 2947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:14:27.380489 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:14:27.380503 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:14:27.380533 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:14:27.382388 (+ 1855us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:14:27.382959 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:14:27.382964 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:14:27.382970 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:14:27.383084 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:14:27.383404 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:14:27.383406 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:14:27.383408 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:14:27.383818 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:14:27.383962 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0805 23:14:27.384381 (+ 419us) server_negotiation.cc:299] Negotiation successful
0805 23:14:27.384735 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":160,"threads_started":1}
I20260805 23:14:27.387512 12278 raft_consensus.cc:348] T 00000000000000000000000000000000 P fae04452d9f048998e94a0ea113588b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fae04452d9f048998e94a0ea113588b8" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 39905 } }
I20260805 23:14:27.387820 12278 raft_consensus.cc:374] T 00000000000000000000000000000000 P fae04452d9f048998e94a0ea113588b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:14:27.387882 12278 raft_consensus.cc:729] T 00000000000000000000000000000000 P fae04452d9f048998e94a0ea113588b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fae04452d9f048998e94a0ea113588b8, State: Initialized, Role: FOLLOWER
I20260805 23:14:27.388337 12278 consensus_queue.cc:260] T 00000000000000000000000000000000 P fae04452d9f048998e94a0ea113588b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fae04452d9f048998e94a0ea113588b8" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 39905 } }
I20260805 23:14:27.388466 12278 raft_consensus.cc:388] T 00000000000000000000000000000000 P fae04452d9f048998e94a0ea113588b8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:14:27.388552 12278 raft_consensus.cc:482] T 00000000000000000000000000000000 P fae04452d9f048998e94a0ea113588b8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:14:27.388650 12278 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fae04452d9f048998e94a0ea113588b8 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:14:27.389750 12278 raft_consensus.cc:504] T 00000000000000000000000000000000 P fae04452d9f048998e94a0ea113588b8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fae04452d9f048998e94a0ea113588b8" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 39905 } }
I20260805 23:14:27.390128 12278 leader_election.cc:302] T 00000000000000000000000000000000 P fae04452d9f048998e94a0ea113588b8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fae04452d9f048998e94a0ea113588b8; no voters:
I20260805 23:14:27.390530 12278 leader_election.cc:288] T 00000000000000000000000000000000 P fae04452d9f048998e94a0ea113588b8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 23:14:27.390632 12283 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fae04452d9f048998e94a0ea113588b8 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:14:27.390880 12283 raft_consensus.cc:686] T 00000000000000000000000000000000 P fae04452d9f048998e94a0ea113588b8 [term 1 LEADER]: Becoming Leader. State: Replica: fae04452d9f048998e94a0ea113588b8, State: Running, Role: LEADER
I20260805 23:14:27.391233 12283 consensus_queue.cc:237] T 00000000000000000000000000000000 P fae04452d9f048998e94a0ea113588b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fae04452d9f048998e94a0ea113588b8" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 39905 } }
I20260805 23:14:27.391881 12278 sys_catalog.cc:564] T 00000000000000000000000000000000 P fae04452d9f048998e94a0ea113588b8 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:14:27.393062 12285 sys_catalog.cc:455] T 00000000000000000000000000000000 P fae04452d9f048998e94a0ea113588b8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fae04452d9f048998e94a0ea113588b8. Latest consensus state: current_term: 1 leader_uuid: "fae04452d9f048998e94a0ea113588b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fae04452d9f048998e94a0ea113588b8" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 39905 } } }
I20260805 23:14:27.393217 12285 sys_catalog.cc:458] T 00000000000000000000000000000000 P fae04452d9f048998e94a0ea113588b8 [sys.catalog]: This master's current role is: LEADER
I20260805 23:14:27.393247 12284 sys_catalog.cc:455] T 00000000000000000000000000000000 P fae04452d9f048998e94a0ea113588b8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fae04452d9f048998e94a0ea113588b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fae04452d9f048998e94a0ea113588b8" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 39905 } } }
I20260805 23:14:27.393347 12284 sys_catalog.cc:458] T 00000000000000000000000000000000 P fae04452d9f048998e94a0ea113588b8 [sys.catalog]: This master's current role is: LEADER
I20260805 23:14:27.394760 12292 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:14:27.397642 12292 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:14:27.403558 12292 catalog_manager.cc:1347] Generated new cluster ID: 9d09e7e1453848ff8e523592a057bed5
I20260805 23:14:27.403643 12292 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:14:27.410838 12292 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:14:27.412042 12292 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:14:27.417459 12292 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fae04452d9f048998e94a0ea113588b8: Generated new TSK 0
I20260805 23:14:27.418227 12292 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.92.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.92.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.92.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.193.keytab.
Entry for principal kudu/127.25.92.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.193.keytab.
I20260805 23:14:27.452708 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971406251336-25971-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971406251336-25971-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971406251336-25971-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971406251336-25971-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.92.193:0
--local_ip_for_outbound_sockets=127.25.92.193
--webserver_interface=127.25.92.193
--webserver_port=0
--tserver_master_addrs=127.25.92.254:39905
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.193.keytab
--principal=kudu/127.25.92.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:33677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:14:27.573977 12305 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:14:27.574268 12305 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:14:27.574314 12305 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:14:27.578581 12305 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:14:27.578670 12305 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:14:27.578732 12305 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.92.193
I20260805 23:14:27.584708 12305 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:33677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971406251336-25971-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971406251336-25971-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.193.keytab
--principal=kudu/127.25.92.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.92.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971406251336-25971-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.92.193
--webserver_port=0
--tserver_master_addrs=127.25.92.254:39905
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12305
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.92.193
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971406251336-25971-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:14:27.585971 12305 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:14:27.587055 12305 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:14:27.597494 12313 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:14:27.597509 12311 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:14:27.597766 12305 server_base.cc:1034] running on GCE node
W20260805 23:14:27.597507 12310 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:14:27.598384 12305 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:14:27.599117 12305 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:14:27.600353 12305 hybrid_clock.cc:648] HybridClock initialized: now 1785971667600320 us; error 48 us; skew 500 ppm
Aug 05 23:14:27 dist-test-slave-ss4n krb5kdc[12194](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971667, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:14:27.603742 12305 init.cc:377] Logged in from keytab as kudu/127.25.92.193@KRBTEST.COM (short username kudu)
I20260805 23:14:27.604979 12305 webserver.cc:460] Webserver started at http://127.25.92.193:35009/ using document root <none> and password file <none>
I20260805 23:14:27.605736 12305 fs_manager.cc:359] Metadata directory not provided
I20260805 23:14:27.605870 12305 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:14:27.606169 12305 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:14:27.608646 12305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971406251336-25971-0/minicluster-data/ts-0/data/instance:
uuid: "faad8cfd923345a48951165a95a200e1"
format_stamp: "Formatted at 2026-08-05 23:14:27 on dist-test-slave-ss4n"
server_key: "c90ed33645590e1abbac54e8eb7384c3"
server_key_iv: "2e515cee0459175bb8b334f59ff5facd"
server_key_version: "encryptionkey@0"
I20260805 23:14:27.609385 12305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971406251336-25971-0/minicluster-data/ts-0/wal/instance:
uuid: "faad8cfd923345a48951165a95a200e1"
format_stamp: "Formatted at 2026-08-05 23:14:27 on dist-test-slave-ss4n"
server_key: "c90ed33645590e1abbac54e8eb7384c3"
server_key_iv: "2e515cee0459175bb8b334f59ff5facd"
server_key_version: "encryptionkey@0"
I20260805 23:14:27.614300 12305 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.000s
I20260805 23:14:27.617439 12320 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:14:27.619339 12305 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260805 23:14:27.619653 12305 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971406251336-25971-0/minicluster-data/ts-0/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971406251336-25971-0/minicluster-data/ts-0/wal
uuid: "faad8cfd923345a48951165a95a200e1"
format_stamp: "Formatted at 2026-08-05 23:14:27 on dist-test-slave-ss4n"
server_key: "c90ed33645590e1abbac54e8eb7384c3"
server_key_iv: "2e515cee0459175bb8b334f59ff5facd"
server_key_version: "encryptionkey@0"
I20260805 23:14:27.619828 12305 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971406251336-25971-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971406251336-25971-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971406251336-25971-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:14:27.634694 12305 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:14:27.638317 12305 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:14:27.638549 12305 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:14:27.639199 12305 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:14:27.640107 12305 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:14:27.640183 12305 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:14:27.640268 12305 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:14:27.640319 12305 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:14:27.650553 12305 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.193:37505
I20260805 23:14:27.650571 12436 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.193:37505 every 8 connection(s)
I20260805 23:14:27.651495 12305 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971406251336-25971-0/minicluster-data/ts-0/data/info.pb
I20260805 23:14:27.660727 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 12305
I20260805 23:14:27.660854 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971406251336-25971-0/minicluster-data/ts-0/wal/instance
I20260805 23:14:27.661183 25971 external_mini_cluster.cc:1428] Setting key e324f91c6f73243091867ec2c159aee9
Aug 05 23:14:27 dist-test-slave-ss4n krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971667, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.193@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM
I20260805 23:14:27.666728 12281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:27.653513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.193:40213 (local address 127.25.92.254:39905)
0805 23:14:27.653643 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0805 23:14:27.653652 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0805 23:14:27.654533 (+ 881us) server_negotiation.cc:407] Connection header received
0805 23:14:27.655987 (+ 1454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:14:27.655991 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:14:27.656054 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:14:27.656164 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:14:27.657381 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:27.658004 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:27.659184 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:27.659421 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:27.659538 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:27.662554 (+ 3016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:14:27.662557 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:14:27.662559 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:14:27.662585 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:14:27.664218 (+ 1633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:14:27.664867 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:14:27.664873 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:14:27.664876 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:14:27.664987 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:14:27.665335 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:14:27.665341 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:14:27.665346 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:14:27.665658 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:14:27.665774 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0805 23:14:27.666307 (+ 533us) server_negotiation.cc:299] Negotiation successful
0805 23:14:27.666460 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260805 23:14:27.667569 12439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:27.653888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:39905 (local address 127.25.92.193:40213)
0805 23:14:27.654366 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:27.654411 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0805 23:14:27.655763 (+ 1352us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:14:27.656373 (+ 610us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:14:27.656380 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:14:27.656738 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:14:27.657223 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:14:27.657238 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:14:27.658143 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:14:27.658159 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:14:27.659039 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:14:27.659049 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:14:27.659584 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:14:27.659588 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:14:27.659763 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:27.660384 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:14:27.660402 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:14:27.662357 (+ 1955us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:14:27.664358 (+ 2001us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:14:27.664369 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:14:27.664388 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:14:27.664752 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:14:27.665090 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:14:27.665092 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:14:27.665093 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:14:27.665237 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:14:27.665775 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:14:27.665781 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:14:27.666050 (+ 269us) client_negotiation.cc:769] Sending connection context
0805 23:14:27.666220 (+ 170us) client_negotiation.cc:240] Negotiation successful
0805 23:14:27.666501 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":123,"threads_started":1}
I20260805 23:14:27.668977 12437 heartbeater.cc:347] Connected to a master server at 127.25.92.254:39905
I20260805 23:14:27.669229 12437 heartbeater.cc:464] Registering TS with master...
I20260805 23:14:27.669840 12437 heartbeater.cc:511] Master 127.25.92.254:39905 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.25.92.194@KRBTEST.COM; defaulting to no policy
I20260805 23:14:27.671548 12241 ts_manager.cc:194] Registered new tserver with Master: faad8cfd923345a48951165a95a200e1 (127.25.92.193:37505)
I20260805 23:14:27.672767 12241 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.92.193@KRBTEST.COM'} at 127.25.92.193:40213
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.92.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.92.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.194.keytab.
Entry for principal kudu/127.25.92.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.194.keytab.
I20260805 23:14:27.692931 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971406251336-25971-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971406251336-25971-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971406251336-25971-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971406251336-25971-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.92.194:0
--local_ip_for_outbound_sockets=127.25.92.194
--webserver_interface=127.25.92.194
--webserver_port=0
--tserver_master_addrs=127.25.92.254:39905
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.194.keytab
--principal=kudu/127.25.92.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:33677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:14:27.811645 12442 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:14:27.811937 12442 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:14:27.811983 12442 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:14:27.816205 12442 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:14:27.816280 12442 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:14:27.816341 12442 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.92.194
I20260805 23:14:27.822749 12442 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:33677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971406251336-25971-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971406251336-25971-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.194.keytab
--principal=kudu/127.25.92.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.92.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971406251336-25971-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.92.194
--webserver_port=0
--tserver_master_addrs=127.25.92.254:39905
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12442
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.92.194
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971406251336-25971-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:14:27.823922 12442 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:14:27.825067 12442 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:14:27.835935 12449 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:14:27.835887 12448 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:14:27.835889 12451 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:14:27.836229 12442 server_base.cc:1034] running on GCE node
I20260805 23:14:27.836854 12442 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:14:27.837671 12442 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:14:27.838867 12442 hybrid_clock.cc:648] HybridClock initialized: now 1785971667838832 us; error 53 us; skew 500 ppm
Aug 05 23:14:27 dist-test-slave-ss4n krb5kdc[12194](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971667, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:14:27.841933 12442 init.cc:377] Logged in from keytab as kudu/127.25.92.194@KRBTEST.COM (short username kudu)
I20260805 23:14:27.842947 12442 webserver.cc:460] Webserver started at http://127.25.92.194:41269/ using document root <none> and password file <none>
I20260805 23:14:27.843600 12442 fs_manager.cc:359] Metadata directory not provided
I20260805 23:14:27.843699 12442 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:14:27.843947 12442 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:14:27.846228 12442 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971406251336-25971-0/minicluster-data/ts-1/data/instance:
uuid: "0cce512cf8c94d27882b5a688f19e1b1"
format_stamp: "Formatted at 2026-08-05 23:14:27 on dist-test-slave-ss4n"
server_key: "c15477669358bfe44a556d1042da4627"
server_key_iv: "52d03173919d0ff0ed902eb99d5bbca8"
server_key_version: "encryptionkey@0"
I20260805 23:14:27.846846 12442 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971406251336-25971-0/minicluster-data/ts-1/wal/instance:
uuid: "0cce512cf8c94d27882b5a688f19e1b1"
format_stamp: "Formatted at 2026-08-05 23:14:27 on dist-test-slave-ss4n"
server_key: "c15477669358bfe44a556d1042da4627"
server_key_iv: "52d03173919d0ff0ed902eb99d5bbca8"
server_key_version: "encryptionkey@0"
I20260805 23:14:27.851068 12442 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260805 23:14:27.853788 12458 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:14:27.855041 12442 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260805 23:14:27.855193 12442 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971406251336-25971-0/minicluster-data/ts-1/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971406251336-25971-0/minicluster-data/ts-1/wal
uuid: "0cce512cf8c94d27882b5a688f19e1b1"
format_stamp: "Formatted at 2026-08-05 23:14:27 on dist-test-slave-ss4n"
server_key: "c15477669358bfe44a556d1042da4627"
server_key_iv: "52d03173919d0ff0ed902eb99d5bbca8"
server_key_version: "encryptionkey@0"
I20260805 23:14:27.855324 12442 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971406251336-25971-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971406251336-25971-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971406251336-25971-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:14:27.877472 12442 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:14:27.880997 12442 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:14:27.881214 12442 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:14:27.881943 12442 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:14:27.882750 12442 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:14:27.882797 12442 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:14:27.882836 12442 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:14:27.882851 12442 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:14:27.892256 12442 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.194:45933
I20260805 23:14:27.892341 12576 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.194:45933 every 8 connection(s)
I20260805 23:14:27.893285 12442 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971406251336-25971-0/minicluster-data/ts-1/data/info.pb
I20260805 23:14:27.900378 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 12442
I20260805 23:14:27.900501 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971406251336-25971-0/minicluster-data/ts-1/wal/instance
I20260805 23:14:27.900835 25971 external_mini_cluster.cc:1428] Setting key eb7e5d4cb97295ce607f473a68f06c0d
Aug 05 23:14:27 dist-test-slave-ss4n krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971667, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.194@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM
I20260805 23:14:27.908134 12281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:27.895441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.194:44617 (local address 127.25.92.254:39905)
0805 23:14:27.895577 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0805 23:14:27.895581 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:14:27.896441 (+ 860us) server_negotiation.cc:407] Connection header received
0805 23:14:27.897811 (+ 1370us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:14:27.897814 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:14:27.897873 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:14:27.897954 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:14:27.899122 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:27.899670 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:27.900989 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:27.901236 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:27.901352 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:27.904179 (+ 2827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:14:27.904183 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:14:27.904185 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:14:27.904211 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:14:27.905768 (+ 1557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:14:27.906330 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:14:27.906334 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:14:27.906335 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:14:27.906411 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:14:27.906764 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:14:27.906768 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:14:27.906769 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:14:27.907008 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:14:27.907138 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0805 23:14:27.907718 (+ 580us) server_negotiation.cc:299] Negotiation successful
0805 23:14:27.907894 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260805 23:14:27.908874 12579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:27.895839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:39905 (local address 127.25.92.194:44617)
0805 23:14:27.896288 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:27.896331 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0805 23:14:27.897604 (+ 1273us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:14:27.898128 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:14:27.898134 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:14:27.898484 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:14:27.898961 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:14:27.898973 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:14:27.899804 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:14:27.899807 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:14:27.900845 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:14:27.900855 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:14:27.901396 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:14:27.901400 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:14:27.901686 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:27.902454 (+ 768us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:14:27.902479 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:14:27.903986 (+ 1507us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:14:27.905894 (+ 1908us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:14:27.905900 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:14:27.905918 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:14:27.906205 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:14:27.906504 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:14:27.906508 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:14:27.906510 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:14:27.906664 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:14:27.907136 (+ 472us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:14:27.907143 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:14:27.907419 (+ 276us) client_negotiation.cc:769] Sending connection context
0805 23:14:27.907624 (+ 205us) client_negotiation.cc:240] Negotiation successful
0805 23:14:27.907915 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":122,"threads_started":1}
I20260805 23:14:27.910169 12577 heartbeater.cc:347] Connected to a master server at 127.25.92.254:39905
I20260805 23:14:27.910408 12577 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.25.92.195@KRBTEST.COM; defaulting to no policy
I20260805 23:14:27.910979 12577 heartbeater.cc:511] Master 127.25.92.254:39905 requested a full tablet report, sending...
I20260805 23:14:27.912099 12241 ts_manager.cc:194] Registered new tserver with Master: 0cce512cf8c94d27882b5a688f19e1b1 (127.25.92.194:45933)
I20260805 23:14:27.912663 12241 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.92.194@KRBTEST.COM'} at 127.25.92.194:44617
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.92.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.92.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.195.keytab.
Entry for principal kudu/127.25.92.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.195.keytab.
I20260805 23:14:27.939095 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971406251336-25971-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971406251336-25971-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971406251336-25971-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971406251336-25971-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.92.195:0
--local_ip_for_outbound_sockets=127.25.92.195
--webserver_interface=127.25.92.195
--webserver_port=0
--tserver_master_addrs=127.25.92.254:39905
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.195.keytab
--principal=kudu/127.25.92.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:33677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:14:28.056208 12582 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:14:28.056529 12582 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:14:28.056623 12582 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:14:28.060948 12582 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:14:28.061059 12582 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:14:28.061156 12582 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.92.195
I20260805 23:14:28.066964 12582 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:33677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971406251336-25971-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971406251336-25971-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.195.keytab
--principal=kudu/127.25.92.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.92.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971406251336-25971-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.92.195
--webserver_port=0
--tserver_master_addrs=127.25.92.254:39905
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12582
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.92.195
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971406251336-25971-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:14:28.068120 12582 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:14:28.069180 12582 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:14:28.079312 12587 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:14:28.079517 12582 server_base.cc:1034] running on GCE node
W20260805 23:14:28.079313 12590 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:14:28.079741 12588 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:14:28.080325 12582 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:14:28.081228 12582 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:14:28.082405 12582 hybrid_clock.cc:648] HybridClock initialized: now 1785971668082368 us; error 57 us; skew 500 ppm
Aug 05 23:14:28 dist-test-slave-ss4n krb5kdc[12194](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971668, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:14:28.085578 12582 init.cc:377] Logged in from keytab as kudu/127.25.92.195@KRBTEST.COM (short username kudu)
I20260805 23:14:28.086612 12582 webserver.cc:460] Webserver started at http://127.25.92.195:42055/ using document root <none> and password file <none>
I20260805 23:14:28.087255 12582 fs_manager.cc:359] Metadata directory not provided
I20260805 23:14:28.087318 12582 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:14:28.087548 12582 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:14:28.089768 12582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971406251336-25971-0/minicluster-data/ts-2/data/instance:
uuid: "5a62db28df7e43aabdcd0c4a7d5c18f7"
format_stamp: "Formatted at 2026-08-05 23:14:28 on dist-test-slave-ss4n"
server_key: "17ab6afb930e83341042e1dd86ebfe0e"
server_key_iv: "0b248333a2877490224d2439234d331a"
server_key_version: "encryptionkey@0"
I20260805 23:14:28.090349 12582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971406251336-25971-0/minicluster-data/ts-2/wal/instance:
uuid: "5a62db28df7e43aabdcd0c4a7d5c18f7"
format_stamp: "Formatted at 2026-08-05 23:14:28 on dist-test-slave-ss4n"
server_key: "17ab6afb930e83341042e1dd86ebfe0e"
server_key_iv: "0b248333a2877490224d2439234d331a"
server_key_version: "encryptionkey@0"
I20260805 23:14:28.094661 12582 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260805 23:14:28.097275 12597 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:14:28.098665 12582 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260805 23:14:28.098800 12582 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971406251336-25971-0/minicluster-data/ts-2/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971406251336-25971-0/minicluster-data/ts-2/wal
uuid: "5a62db28df7e43aabdcd0c4a7d5c18f7"
format_stamp: "Formatted at 2026-08-05 23:14:28 on dist-test-slave-ss4n"
server_key: "17ab6afb930e83341042e1dd86ebfe0e"
server_key_iv: "0b248333a2877490224d2439234d331a"
server_key_version: "encryptionkey@0"
I20260805 23:14:28.098922 12582 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971406251336-25971-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971406251336-25971-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971406251336-25971-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:14:28.118739 12582 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:14:28.122426 12582 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:14:28.122776 12582 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:14:28.123485 12582 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:14:28.124508 12582 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:14:28.124567 12582 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:14:28.124635 12582 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:14:28.124673 12582 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:14:28.134617 12582 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.195:41567
I20260805 23:14:28.134637 12712 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.195:41567 every 8 connection(s)
I20260805 23:14:28.135604 12582 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971406251336-25971-0/minicluster-data/ts-2/data/info.pb
I20260805 23:14:28.137395 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 12582
I20260805 23:14:28.137609 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971406251336-25971-0/minicluster-data/ts-2/wal/instance
I20260805 23:14:28.138046 25971 external_mini_cluster.cc:1428] Setting key 3d8140d1b924a91e3a68cbf7acc1d424
Aug 05 23:14:28 dist-test-slave-ss4n krb5kdc[12194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971668, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.195@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM
I20260805 23:14:28.151589 12281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:28.137878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.195:45285 (local address 127.25.92.254:39905)
0805 23:14:28.138089 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0805 23:14:28.138095 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:14:28.139537 (+ 1442us) server_negotiation.cc:407] Connection header received
0805 23:14:28.140583 (+ 1046us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:14:28.140586 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:14:28.140640 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:14:28.140712 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:14:28.142065 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:28.142677 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:28.144003 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:28.144247 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:28.144338 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:28.147227 (+ 2889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:14:28.147230 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:14:28.147232 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:14:28.147260 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:14:28.148907 (+ 1647us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:14:28.149609 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:14:28.149613 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:14:28.149614 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:14:28.149707 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:14:28.150139 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:14:28.150142 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:14:28.150144 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:14:28.150410 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:14:28.150574 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0805 23:14:28.151243 (+ 669us) server_negotiation.cc:299] Negotiation successful
0805 23:14:28.151403 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260805 23:14:28.152655 12715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:28.138365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:39905 (local address 127.25.92.195:45285)
0805 23:14:28.139370 (+ 1005us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:28.139431 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0805 23:14:28.140381 (+ 950us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:14:28.140873 (+ 492us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:14:28.140880 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:14:28.141288 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:14:28.141911 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:14:28.141927 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:14:28.142828 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:14:28.142831 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:14:28.143871 (+ 1040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:14:28.143881 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:14:28.144453 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:14:28.144457 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:14:28.144660 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:28.145308 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:14:28.145325 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:14:28.147015 (+ 1690us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:14:28.149064 (+ 2049us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:14:28.149070 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:14:28.149085 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:14:28.149393 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:14:28.149839 (+ 446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:14:28.149841 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:14:28.149843 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:14:28.150014 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:14:28.150598 (+ 584us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:14:28.150607 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:14:28.150946 (+ 339us) client_negotiation.cc:769] Sending connection context
0805 23:14:28.151175 (+ 229us) client_negotiation.cc:240] Negotiation successful
0805 23:14:28.151546 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":781,"thread_start_us":624,"threads_started":1}
I20260805 23:14:28.153951 12713 heartbeater.cc:347] Connected to a master server at 127.25.92.254:39905
I20260805 23:14:28.154230 12713 heartbeater.cc:464] Registering TS with master...
I20260805 23:14:28.154771 12713 heartbeater.cc:511] Master 127.25.92.254:39905 requested a full tablet report, sending...
I20260805 23:14:28.155889 12241 ts_manager.cc:194] Registered new tserver with Master: 5a62db28df7e43aabdcd0c4a7d5c18f7 (127.25.92.195:41567)
I20260805 23:14:28.156606 12241 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.92.195@KRBTEST.COM'} at 127.25.92.195:45285
I20260805 23:14:28.164870 25971 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260805 23:14:28.177024 12281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:28.167345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34220 (local address 127.25.92.254:39905)
0805 23:14:28.167584 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0805 23:14:28.167588 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:14:28.168426 (+ 838us) server_negotiation.cc:407] Connection header received
0805 23:14:28.168548 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:14:28.168550 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:14:28.168600 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:14:28.168691 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:14:28.169542 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:28.170139 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:28.171069 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:28.171310 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:28.171400 (+ 90us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:28.173167 (+ 1767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:14:28.173170 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:14:28.173173 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:14:28.173200 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:14:28.174970 (+ 1770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:14:28.175581 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:14:28.175584 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:14:28.175585 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:14:28.175668 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:14:28.176044 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:14:28.176047 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:14:28.176048 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:14:28.176314 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:14:28.176442 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0805 23:14:28.176800 (+ 358us) server_negotiation.cc:299] Negotiation successful
0805 23:14:28.176890 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260805 23:14:28.180837 12241 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34220:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260805 23:14:28.184085 12241 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260805 23:14:28.200639 12726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:28.194181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.195:41567 (local address 127.0.0.1:36528)
0805 23:14:28.194446 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:28.194461 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0805 23:14:28.194556 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:14:28.195952 (+ 1396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:14:28.195955 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:14:28.195983 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:14:28.196200 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:14:28.196209 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:14:28.197415 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:14:28.197420 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:14:28.198911 (+ 1491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:14:28.198922 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:14:28.200129 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:14:28.200132 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:14:28.200246 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:28.200277 (+ 31us) client_negotiation.cc:769] Sending connection context
0805 23:14:28.200327 (+ 50us) client_negotiation.cc:240] Negotiation successful
0805 23:14:28.200449 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"mutex_wait_us":174,"spinlock_wait_cycles":5248,"thread_start_us":109,"threads_started":1}
I20260805 23:14:28.200820 12724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:28.193677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.193:37505 (local address 127.0.0.1:36270)
0805 23:14:28.194285 (+ 608us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:28.194338 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0805 23:14:28.194478 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:14:28.195908 (+ 1430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:14:28.195945 (+ 37us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:14:28.195985 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:14:28.196167 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:14:28.196179 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:14:28.197667 (+ 1488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:14:28.197672 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:14:28.199173 (+ 1501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:14:28.199183 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:14:28.200301 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:14:28.200306 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:14:28.200412 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:28.200436 (+ 24us) client_negotiation.cc:769] Sending connection context
0805 23:14:28.200528 (+ 92us) client_negotiation.cc:240] Negotiation successful
0805 23:14:28.200598 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"mutex_wait_us":87,"spinlock_wait_cycles":7168,"thread_start_us":110,"threads_started":1}
I20260805 23:14:28.200986 12725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:28.193815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36270 (local address 127.25.92.193:37505)
0805 23:14:28.195169 (+ 1354us) server_negotiation.cc:206] Beginning negotiation
0805 23:14:28.195178 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0805 23:14:28.195207 (+ 29us) server_negotiation.cc:407] Connection header received
0805 23:14:28.195317 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:14:28.195323 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:14:28.195566 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:14:28.195799 (+ 233us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:14:28.196304 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:28.197507 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:28.199299 (+ 1792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:28.200154 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:28.200284 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:28.200434 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0805 23:14:28.200550 (+ 116us) server_negotiation.cc:299] Negotiation successful
0805 23:14:28.200645 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1217,"thread_start_us":155,"threads_started":1}
I20260805 23:14:28.201342 12729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:28.195117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50960 (local address 127.25.92.194:45933)
0805 23:14:28.195458 (+ 341us) server_negotiation.cc:206] Beginning negotiation
0805 23:14:28.195470 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0805 23:14:28.195617 (+ 147us) server_negotiation.cc:407] Connection header received
0805 23:14:28.195742 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:14:28.195747 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:14:28.195930 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:14:28.196126 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:14:28.196524 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:28.197629 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:28.199188 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:28.199871 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:28.200007 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:28.200180 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0805 23:14:28.200445 (+ 265us) server_negotiation.cc:299] Negotiation successful
0805 23:14:28.200570 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":239,"thread_start_us":128,"threads_started":1}
I20260805 23:14:28.201797 12727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:28.194496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.194:45933 (local address 127.0.0.1:50960)
0805 23:14:28.195511 (+ 1015us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:28.195526 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0805 23:14:28.195624 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:14:28.196287 (+ 663us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:14:28.196290 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:14:28.196311 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:14:28.196389 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:14:28.196399 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:14:28.197765 (+ 1366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:14:28.197768 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:14:28.199038 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:14:28.199056 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:14:28.200012 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:14:28.200016 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:14:28.200134 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:28.200178 (+ 44us) client_negotiation.cc:769] Sending connection context
0805 23:14:28.200298 (+ 120us) client_negotiation.cc:240] Negotiation successful
0805 23:14:28.201625 (+ 1327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":953,"mutex_wait_us":1218,"thread_start_us":102,"threads_started":1}
I20260805 23:14:28.202427 12728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:28.194804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36528 (local address 127.25.92.195:41567)
0805 23:14:28.195458 (+ 654us) server_negotiation.cc:206] Beginning negotiation
0805 23:14:28.195470 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0805 23:14:28.195493 (+ 23us) server_negotiation.cc:407] Connection header received
0805 23:14:28.195586 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:14:28.195591 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:14:28.195800 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:14:28.196051 (+ 251us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:14:28.196350 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:28.197225 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:28.199055 (+ 1830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:28.199970 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:28.201061 (+ 1091us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:28.202037 (+ 976us) server_negotiation.cc:1035] Waiting for connection context
0805 23:14:28.202113 (+ 76us) server_negotiation.cc:299] Negotiation successful
0805 23:14:28.202257 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":541,"thread_start_us":191,"threads_started":1}
I20260805 23:14:28.204418 12368 tablet_service.cc:1467] Processing CreateTablet for tablet 70da09dabcf04b0c84fc877c804dd94d (DEFAULT_TABLE table=test-table [id=a35af753d91c4b1aacf2067d50b814bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:14:28.204943 12646 tablet_service.cc:1467] Processing CreateTablet for tablet 70da09dabcf04b0c84fc877c804dd94d (DEFAULT_TABLE table=test-table [id=a35af753d91c4b1aacf2067d50b814bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:14:28.204943 12507 tablet_service.cc:1467] Processing CreateTablet for tablet 70da09dabcf04b0c84fc877c804dd94d (DEFAULT_TABLE table=test-table [id=a35af753d91c4b1aacf2067d50b814bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:14:28.205567 12368 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 70da09dabcf04b0c84fc877c804dd94d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:14:28.205926 12646 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 70da09dabcf04b0c84fc877c804dd94d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:14:28.205926 12507 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 70da09dabcf04b0c84fc877c804dd94d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:14:28.212332 12730 tablet_bootstrap.cc:492] T 70da09dabcf04b0c84fc877c804dd94d P faad8cfd923345a48951165a95a200e1: Bootstrap starting.
I20260805 23:14:28.213795 12732 tablet_bootstrap.cc:492] T 70da09dabcf04b0c84fc877c804dd94d P 5a62db28df7e43aabdcd0c4a7d5c18f7: Bootstrap starting.
I20260805 23:14:28.214823 12730 tablet_bootstrap.cc:654] T 70da09dabcf04b0c84fc877c804dd94d P faad8cfd923345a48951165a95a200e1: Neither blocks nor log segments found. Creating new log.
I20260805 23:14:28.215554 12732 tablet_bootstrap.cc:654] T 70da09dabcf04b0c84fc877c804dd94d P 5a62db28df7e43aabdcd0c4a7d5c18f7: Neither blocks nor log segments found. Creating new log.
I20260805 23:14:28.215646 12730 log.cc:824] T 70da09dabcf04b0c84fc877c804dd94d P faad8cfd923345a48951165a95a200e1: Log is configured to *not* fsync() on all Append() calls
I20260805 23:14:28.216838 12732 log.cc:824] T 70da09dabcf04b0c84fc877c804dd94d P 5a62db28df7e43aabdcd0c4a7d5c18f7: Log is configured to *not* fsync() on all Append() calls
I20260805 23:14:28.216888 12731 tablet_bootstrap.cc:492] T 70da09dabcf04b0c84fc877c804dd94d P 0cce512cf8c94d27882b5a688f19e1b1: Bootstrap starting.
I20260805 23:14:28.218271 12730 tablet_bootstrap.cc:492] T 70da09dabcf04b0c84fc877c804dd94d P faad8cfd923345a48951165a95a200e1: No bootstrap required, opened a new log
I20260805 23:14:28.218568 12730 ts_tablet_manager.cc:1397] T 70da09dabcf04b0c84fc877c804dd94d P faad8cfd923345a48951165a95a200e1: Time spent bootstrapping tablet: real 0.006s user 0.006s sys 0.000s
I20260805 23:14:28.218976 12732 tablet_bootstrap.cc:492] T 70da09dabcf04b0c84fc877c804dd94d P 5a62db28df7e43aabdcd0c4a7d5c18f7: No bootstrap required, opened a new log
I20260805 23:14:28.219174 12732 ts_tablet_manager.cc:1397] T 70da09dabcf04b0c84fc877c804dd94d P 5a62db28df7e43aabdcd0c4a7d5c18f7: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260805 23:14:28.219254 12731 tablet_bootstrap.cc:654] T 70da09dabcf04b0c84fc877c804dd94d P 0cce512cf8c94d27882b5a688f19e1b1: Neither blocks nor log segments found. Creating new log.
I20260805 23:14:28.220367 12731 log.cc:824] T 70da09dabcf04b0c84fc877c804dd94d P 0cce512cf8c94d27882b5a688f19e1b1: Log is configured to *not* fsync() on all Append() calls
I20260805 23:14:28.222203 12731 tablet_bootstrap.cc:492] T 70da09dabcf04b0c84fc877c804dd94d P 0cce512cf8c94d27882b5a688f19e1b1: No bootstrap required, opened a new log
I20260805 23:14:28.222407 12731 ts_tablet_manager.cc:1397] T 70da09dabcf04b0c84fc877c804dd94d P 0cce512cf8c94d27882b5a688f19e1b1: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260805 23:14:28.223335 12732 raft_consensus.cc:348] T 70da09dabcf04b0c84fc877c804dd94d P 5a62db28df7e43aabdcd0c4a7d5c18f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "faad8cfd923345a48951165a95a200e1" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 37505 } } peers { permanent_uuid: "0cce512cf8c94d27882b5a688f19e1b1" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 45933 } } peers { permanent_uuid: "5a62db28df7e43aabdcd0c4a7d5c18f7" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 41567 } }
I20260805 23:14:28.223335 12730 raft_consensus.cc:348] T 70da09dabcf04b0c84fc877c804dd94d P faad8cfd923345a48951165a95a200e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "faad8cfd923345a48951165a95a200e1" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 37505 } } peers { permanent_uuid: "0cce512cf8c94d27882b5a688f19e1b1" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 45933 } } peers { permanent_uuid: "5a62db28df7e43aabdcd0c4a7d5c18f7" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 41567 } }
I20260805 23:14:28.223635 12732 raft_consensus.cc:374] T 70da09dabcf04b0c84fc877c804dd94d P 5a62db28df7e43aabdcd0c4a7d5c18f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:14:28.223655 12730 raft_consensus.cc:374] T 70da09dabcf04b0c84fc877c804dd94d P faad8cfd923345a48951165a95a200e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:14:28.223695 12732 raft_consensus.cc:729] T 70da09dabcf04b0c84fc877c804dd94d P 5a62db28df7e43aabdcd0c4a7d5c18f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a62db28df7e43aabdcd0c4a7d5c18f7, State: Initialized, Role: FOLLOWER
I20260805 23:14:28.223701 12730 raft_consensus.cc:729] T 70da09dabcf04b0c84fc877c804dd94d P faad8cfd923345a48951165a95a200e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: faad8cfd923345a48951165a95a200e1, State: Initialized, Role: FOLLOWER
I20260805 23:14:28.224202 12732 consensus_queue.cc:260] T 70da09dabcf04b0c84fc877c804dd94d P 5a62db28df7e43aabdcd0c4a7d5c18f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "faad8cfd923345a48951165a95a200e1" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 37505 } } peers { permanent_uuid: "0cce512cf8c94d27882b5a688f19e1b1" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 45933 } } peers { permanent_uuid: "5a62db28df7e43aabdcd0c4a7d5c18f7" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 41567 } }
I20260805 23:14:28.224186 12730 consensus_queue.cc:260] T 70da09dabcf04b0c84fc877c804dd94d P faad8cfd923345a48951165a95a200e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "faad8cfd923345a48951165a95a200e1" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 37505 } } peers { permanent_uuid: "0cce512cf8c94d27882b5a688f19e1b1" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 45933 } } peers { permanent_uuid: "5a62db28df7e43aabdcd0c4a7d5c18f7" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 41567 } }
I20260805 23:14:28.225076 12713 heartbeater.cc:503] Master 127.25.92.254:39905 was elected leader, sending a full tablet report...
I20260805 23:14:28.225287 12732 ts_tablet_manager.cc:1428] T 70da09dabcf04b0c84fc877c804dd94d P 5a62db28df7e43aabdcd0c4a7d5c18f7: Time spent starting tablet: real 0.006s user 0.006s sys 0.002s
I20260805 23:14:28.225287 12730 ts_tablet_manager.cc:1428] T 70da09dabcf04b0c84fc877c804dd94d P faad8cfd923345a48951165a95a200e1: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260805 23:14:28.225546 12437 heartbeater.cc:503] Master 127.25.92.254:39905 was elected leader, sending a full tablet report...
I20260805 23:14:28.228404 12731 raft_consensus.cc:348] T 70da09dabcf04b0c84fc877c804dd94d P 0cce512cf8c94d27882b5a688f19e1b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "faad8cfd923345a48951165a95a200e1" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 37505 } } peers { permanent_uuid: "0cce512cf8c94d27882b5a688f19e1b1" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 45933 } } peers { permanent_uuid: "5a62db28df7e43aabdcd0c4a7d5c18f7" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 41567 } }
I20260805 23:14:28.228731 12731 raft_consensus.cc:374] T 70da09dabcf04b0c84fc877c804dd94d P 0cce512cf8c94d27882b5a688f19e1b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:14:28.228811 12731 raft_consensus.cc:729] T 70da09dabcf04b0c84fc877c804dd94d P 0cce512cf8c94d27882b5a688f19e1b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cce512cf8c94d27882b5a688f19e1b1, State: Initialized, Role: FOLLOWER
I20260805 23:14:28.229353 12731 consensus_queue.cc:260] T 70da09dabcf04b0c84fc877c804dd94d P 0cce512cf8c94d27882b5a688f19e1b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "faad8cfd923345a48951165a95a200e1" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 37505 } } peers { permanent_uuid: "0cce512cf8c94d27882b5a688f19e1b1" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 45933 } } peers { permanent_uuid: "5a62db28df7e43aabdcd0c4a7d5c18f7" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 41567 } }
I20260805 23:14:28.230440 12731 ts_tablet_manager.cc:1428] T 70da09dabcf04b0c84fc877c804dd94d P 0cce512cf8c94d27882b5a688f19e1b1: Time spent starting tablet: real 0.008s user 0.003s sys 0.004s
I20260805 23:14:28.230609 12577 heartbeater.cc:503] Master 127.25.92.254:39905 was elected leader, sending a full tablet report...
I20260805 23:14:28.235499 12738 raft_consensus.cc:482] T 70da09dabcf04b0c84fc877c804dd94d P 0cce512cf8c94d27882b5a688f19e1b1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 23:14:28.235646 12738 raft_consensus.cc:504] T 70da09dabcf04b0c84fc877c804dd94d P 0cce512cf8c94d27882b5a688f19e1b1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "faad8cfd923345a48951165a95a200e1" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 37505 } } peers { permanent_uuid: "0cce512cf8c94d27882b5a688f19e1b1" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 45933 } } peers { permanent_uuid: "5a62db28df7e43aabdcd0c4a7d5c18f7" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 41567 } }
I20260805 23:14:28.236672 12738 leader_election.cc:288] T 70da09dabcf04b0c84fc877c804dd94d P 0cce512cf8c94d27882b5a688f19e1b1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers faad8cfd923345a48951165a95a200e1 (127.25.92.193:37505), 5a62db28df7e43aabdcd0c4a7d5c18f7 (127.25.92.195:41567)
I20260805 23:14:28.240381 12725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:28.237111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.194:39721 (local address 127.25.92.193:37505)
0805 23:14:28.237327 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0805 23:14:28.237330 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 23:14:28.237340 (+ 10us) server_negotiation.cc:407] Connection header received
0805 23:14:28.237381 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:14:28.237383 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:14:28.237422 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:14:28.237567 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:14:28.237836 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:28.238339 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:28.239378 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:28.239806 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:28.239941 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:28.239996 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0805 23:14:28.240185 (+ 189us) server_negotiation.cc:299] Negotiation successful
0805 23:14:28.240258 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152}
I20260805 23:14:28.240381 12579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:28.236835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.193:37505 (local address 127.25.92.194:39721)
0805 23:14:28.236942 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:28.236959 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0805 23:14:28.237047 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:14:28.237572 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:14:28.237575 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:14:28.237593 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:14:28.237715 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:14:28.237723 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:14:28.238452 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:14:28.238457 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:14:28.239248 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:14:28.239257 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:14:28.239931 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:14:28.239935 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:14:28.240041 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:28.240071 (+ 30us) client_negotiation.cc:769] Sending connection context
0805 23:14:28.240169 (+ 98us) client_negotiation.cc:240] Negotiation successful
0805 23:14:28.240247 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260805 23:14:28.241045 12739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:28.236987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.195:41567 (local address 127.25.92.194:47697)
0805 23:14:28.237305 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:28.237320 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0805 23:14:28.237451 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:14:28.237755 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:14:28.237758 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:14:28.237777 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:14:28.237922 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:14:28.237928 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:14:28.238759 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:14:28.238766 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:14:28.239634 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:14:28.239642 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:14:28.240378 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:14:28.240382 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:14:28.240694 (+ 312us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:28.240720 (+ 26us) client_negotiation.cc:769] Sending connection context
0805 23:14:28.240844 (+ 124us) client_negotiation.cc:240] Negotiation successful
0805 23:14:28.240926 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":144,"threads_started":1}
I20260805 23:14:28.241071 12728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:28.237095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.194:47697 (local address 127.25.92.195:41567)
0805 23:14:28.237249 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0805 23:14:28.237253 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:14:28.237410 (+ 157us) server_negotiation.cc:407] Connection header received
0805 23:14:28.237584 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:14:28.237587 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:14:28.237645 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:14:28.237732 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:14:28.238046 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:28.238638 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:28.239765 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:28.240246 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:28.240389 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:28.240440 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0805 23:14:28.240877 (+ 437us) server_negotiation.cc:299] Negotiation successful
0805 23:14:28.240962 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260805 23:14:28.241183 12390 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70da09dabcf04b0c84fc877c804dd94d" candidate_uuid: "0cce512cf8c94d27882b5a688f19e1b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "faad8cfd923345a48951165a95a200e1" is_pre_election: true
I20260805 23:14:28.241552 12390 raft_consensus.cc:2454] T 70da09dabcf04b0c84fc877c804dd94d P faad8cfd923345a48951165a95a200e1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0cce512cf8c94d27882b5a688f19e1b1 in term 0.
I20260805 23:14:28.242079 12667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70da09dabcf04b0c84fc877c804dd94d" candidate_uuid: "0cce512cf8c94d27882b5a688f19e1b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a62db28df7e43aabdcd0c4a7d5c18f7" is_pre_election: true
I20260805 23:14:28.242205 12460 leader_election.cc:302] T 70da09dabcf04b0c84fc877c804dd94d P 0cce512cf8c94d27882b5a688f19e1b1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0cce512cf8c94d27882b5a688f19e1b1, faad8cfd923345a48951165a95a200e1; no voters:
I20260805 23:14:28.242409 12667 raft_consensus.cc:2454] T 70da09dabcf04b0c84fc877c804dd94d P 5a62db28df7e43aabdcd0c4a7d5c18f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0cce512cf8c94d27882b5a688f19e1b1 in term 0.
I20260805 23:14:28.242604 12738 raft_consensus.cc:2781] T 70da09dabcf04b0c84fc877c804dd94d P 0cce512cf8c94d27882b5a688f19e1b1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260805 23:14:28.242969 12738 raft_consensus.cc:482] T 70da09dabcf04b0c84fc877c804dd94d P 0cce512cf8c94d27882b5a688f19e1b1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260805 23:14:28.243046 12738 raft_consensus.cc:3037] T 70da09dabcf04b0c84fc877c804dd94d P 0cce512cf8c94d27882b5a688f19e1b1 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:14:28.244383 12738 raft_consensus.cc:504] T 70da09dabcf04b0c84fc877c804dd94d P 0cce512cf8c94d27882b5a688f19e1b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "faad8cfd923345a48951165a95a200e1" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 37505 } } peers { permanent_uuid: "0cce512cf8c94d27882b5a688f19e1b1" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 45933 } } peers { permanent_uuid: "5a62db28df7e43aabdcd0c4a7d5c18f7" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 41567 } }
I20260805 23:14:28.244808 12738 leader_election.cc:288] T 70da09dabcf04b0c84fc877c804dd94d P 0cce512cf8c94d27882b5a688f19e1b1 [CANDIDATE]: Term 1 election: Requested vote from peers faad8cfd923345a48951165a95a200e1 (127.25.92.193:37505), 5a62db28df7e43aabdcd0c4a7d5c18f7 (127.25.92.195:41567)
I20260805 23:14:28.245244 12390 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70da09dabcf04b0c84fc877c804dd94d" candidate_uuid: "0cce512cf8c94d27882b5a688f19e1b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "faad8cfd923345a48951165a95a200e1"
I20260805 23:14:28.245270 12667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70da09dabcf04b0c84fc877c804dd94d" candidate_uuid: "0cce512cf8c94d27882b5a688f19e1b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a62db28df7e43aabdcd0c4a7d5c18f7"
I20260805 23:14:28.245373 12390 raft_consensus.cc:3037] T 70da09dabcf04b0c84fc877c804dd94d P faad8cfd923345a48951165a95a200e1 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:14:28.245411 12667 raft_consensus.cc:3037] T 70da09dabcf04b0c84fc877c804dd94d P 5a62db28df7e43aabdcd0c4a7d5c18f7 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:14:28.246562 12667 raft_consensus.cc:2454] T 70da09dabcf04b0c84fc877c804dd94d P 5a62db28df7e43aabdcd0c4a7d5c18f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0cce512cf8c94d27882b5a688f19e1b1 in term 1.
I20260805 23:14:28.246551 12390 raft_consensus.cc:2454] T 70da09dabcf04b0c84fc877c804dd94d P faad8cfd923345a48951165a95a200e1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0cce512cf8c94d27882b5a688f19e1b1 in term 1.
I20260805 23:14:28.246945 12462 leader_election.cc:302] T 70da09dabcf04b0c84fc877c804dd94d P 0cce512cf8c94d27882b5a688f19e1b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0cce512cf8c94d27882b5a688f19e1b1, 5a62db28df7e43aabdcd0c4a7d5c18f7; no voters:
I20260805 23:14:28.247193 12738 raft_consensus.cc:2781] T 70da09dabcf04b0c84fc877c804dd94d P 0cce512cf8c94d27882b5a688f19e1b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:14:28.247466 12738 raft_consensus.cc:686] T 70da09dabcf04b0c84fc877c804dd94d P 0cce512cf8c94d27882b5a688f19e1b1 [term 1 LEADER]: Becoming Leader. State: Replica: 0cce512cf8c94d27882b5a688f19e1b1, State: Running, Role: LEADER
I20260805 23:14:28.247875 12738 consensus_queue.cc:237] T 70da09dabcf04b0c84fc877c804dd94d P 0cce512cf8c94d27882b5a688f19e1b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "faad8cfd923345a48951165a95a200e1" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 37505 } } peers { permanent_uuid: "0cce512cf8c94d27882b5a688f19e1b1" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 45933 } } peers { permanent_uuid: "5a62db28df7e43aabdcd0c4a7d5c18f7" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 41567 } }
I20260805 23:14:28.251439 12240 catalog_manager.cc:5495] T 70da09dabcf04b0c84fc877c804dd94d P 0cce512cf8c94d27882b5a688f19e1b1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0cce512cf8c94d27882b5a688f19e1b1 (127.25.92.194). New cstate: current_term: 1 leader_uuid: "0cce512cf8c94d27882b5a688f19e1b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "faad8cfd923345a48951165a95a200e1" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 37505 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0cce512cf8c94d27882b5a688f19e1b1" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 45933 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a62db28df7e43aabdcd0c4a7d5c18f7" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 41567 } health_report { overall_health: UNKNOWN } } }
I20260805 23:14:28.266352 12729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:28.262484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50970 (local address 127.25.92.194:45933)
0805 23:14:28.262630 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0805 23:14:28.262634 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:14:28.262675 (+ 41us) server_negotiation.cc:407] Connection header received
0805 23:14:28.262760 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:14:28.262763 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:14:28.262820 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:14:28.262911 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 23:14:28.263142 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:28.263727 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:28.264569 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:28.264809 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:28.264898 (+ 89us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:28.265326 (+ 428us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 23:14:28.265899 (+ 573us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 23:14:28.266027 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0805 23:14:28.266173 (+ 146us) server_negotiation.cc:299] Negotiation successful
0805 23:14:28.266214 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260805 23:14:28.273118 12667 raft_consensus.cc:1261] T 70da09dabcf04b0c84fc877c804dd94d P 5a62db28df7e43aabdcd0c4a7d5c18f7 [term 1 FOLLOWER]: Refusing update from remote peer 0cce512cf8c94d27882b5a688f19e1b1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:14:28.273115 12390 raft_consensus.cc:1261] T 70da09dabcf04b0c84fc877c804dd94d P faad8cfd923345a48951165a95a200e1 [term 1 FOLLOWER]: Refusing update from remote peer 0cce512cf8c94d27882b5a688f19e1b1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:14:28.274113 12740 consensus_queue.cc:1035] T 70da09dabcf04b0c84fc877c804dd94d P 0cce512cf8c94d27882b5a688f19e1b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a62db28df7e43aabdcd0c4a7d5c18f7" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 41567 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:14:28.274340 12738 consensus_queue.cc:1035] T 70da09dabcf04b0c84fc877c804dd94d P 0cce512cf8c94d27882b5a688f19e1b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "faad8cfd923345a48951165a95a200e1" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 37505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:14:28.283598 12743 mvcc.cc:205] Tried to move back new op lower bound from 7315339953237602304 to 7315339953146269696. Current Snapshot: MvccSnapshot[applied={T|T < 7315339953237602304}]
I20260805 23:14:28.284125 12745 mvcc.cc:205] Tried to move back new op lower bound from 7315339953237602304 to 7315339953146269696. Current Snapshot: MvccSnapshot[applied={T|T < 7315339953237602304}]
I20260805 23:14:28.283785 12744 mvcc.cc:205] Tried to move back new op lower bound from 7315339953237602304 to 7315339953146269696. Current Snapshot: MvccSnapshot[applied={T|T < 7315339953237602304}]
I20260805 23:14:28.292346 12241 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34220:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:14:28.292565 12241 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34220:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:14:28.295603 12241 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fae04452d9f048998e94a0ea113588b8: Sending DeleteTablet for 3 replicas of tablet 70da09dabcf04b0c84fc877c804dd94d
I20260805 23:14:28.296628 12507 tablet_service.cc:1514] Processing DeleteTablet for tablet 70da09dabcf04b0c84fc877c804dd94d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:14:28 UTC) from {username='kudu', principal='kudu/127.25.92.254@KRBTEST.COM'} at 127.0.0.1:50960
I20260805 23:14:28.296656 12368 tablet_service.cc:1514] Processing DeleteTablet for tablet 70da09dabcf04b0c84fc877c804dd94d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:14:28 UTC) from {username='kudu', principal='kudu/127.25.92.254@KRBTEST.COM'} at 127.0.0.1:36270
I20260805 23:14:28.297230 12646 tablet_service.cc:1514] Processing DeleteTablet for tablet 70da09dabcf04b0c84fc877c804dd94d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:14:28 UTC) from {username='kudu', principal='kudu/127.25.92.254@KRBTEST.COM'} at 127.0.0.1:36528
I20260805 23:14:28.297519 12753 tablet_replica.cc:330] T 70da09dabcf04b0c84fc877c804dd94d P faad8cfd923345a48951165a95a200e1: stopping tablet replica
I20260805 23:14:28.297758 12754 tablet_replica.cc:330] T 70da09dabcf04b0c84fc877c804dd94d P 5a62db28df7e43aabdcd0c4a7d5c18f7: stopping tablet replica
I20260805 23:14:28.297920 12753 raft_consensus.cc:2229] T 70da09dabcf04b0c84fc877c804dd94d P faad8cfd923345a48951165a95a200e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 23:14:28.298060 12754 raft_consensus.cc:2229] T 70da09dabcf04b0c84fc877c804dd94d P 5a62db28df7e43aabdcd0c4a7d5c18f7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 23:14:28.298302 12753 raft_consensus.cc:2258] T 70da09dabcf04b0c84fc877c804dd94d P faad8cfd923345a48951165a95a200e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:14:28.298328 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 12305
I20260805 23:14:28.298364 12754 raft_consensus.cc:2258] T 70da09dabcf04b0c84fc877c804dd94d P 5a62db28df7e43aabdcd0c4a7d5c18f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:14:28.302109 12754 ts_tablet_manager.cc:1905] T 70da09dabcf04b0c84fc877c804dd94d P 5a62db28df7e43aabdcd0c4a7d5c18f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:14:28.297241 12752 tablet_replica.cc:330] T 70da09dabcf04b0c84fc877c804dd94d P 0cce512cf8c94d27882b5a688f19e1b1: stopping tablet replica
I20260805 23:14:28.302639 12752 raft_consensus.cc:2229] T 70da09dabcf04b0c84fc877c804dd94d P 0cce512cf8c94d27882b5a688f19e1b1 [term 1 LEADER]: Raft consensus shutting down.
I20260805 23:14:28.303117 12752 raft_consensus.cc:2258] T 70da09dabcf04b0c84fc877c804dd94d P 0cce512cf8c94d27882b5a688f19e1b1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:14:28.304926 12752 ts_tablet_manager.cc:1905] T 70da09dabcf04b0c84fc877c804dd94d P 0cce512cf8c94d27882b5a688f19e1b1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:14:28.305527 12754 ts_tablet_manager.cc:1918] T 70da09dabcf04b0c84fc877c804dd94d P 5a62db28df7e43aabdcd0c4a7d5c18f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 23:14:28.305632 12754 log.cc:1196] T 70da09dabcf04b0c84fc877c804dd94d P 5a62db28df7e43aabdcd0c4a7d5c18f7: Deleting WAL directory at /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971406251336-25971-0/minicluster-data/ts-2/wal/wals/70da09dabcf04b0c84fc877c804dd94d
I20260805 23:14:28.306282 12754 ts_tablet_manager.cc:1939] T 70da09dabcf04b0c84fc877c804dd94d P 5a62db28df7e43aabdcd0c4a7d5c18f7: Deleting consensus metadata
W20260805 23:14:28.307484 12226 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.25.92.193:37505 (error 108)
I20260805 23:14:28.308050 12228 catalog_manager.cc:4841] TS 5a62db28df7e43aabdcd0c4a7d5c18f7 (127.25.92.195:41567): tablet 70da09dabcf04b0c84fc877c804dd94d (table test-table [id=a35af753d91c4b1aacf2067d50b814bf]) successfully deleted
I20260805 23:14:28.308079 12752 ts_tablet_manager.cc:1918] T 70da09dabcf04b0c84fc877c804dd94d P 0cce512cf8c94d27882b5a688f19e1b1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 23:14:28.308197 12752 log.cc:1196] T 70da09dabcf04b0c84fc877c804dd94d P 0cce512cf8c94d27882b5a688f19e1b1: Deleting WAL directory at /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971406251336-25971-0/minicluster-data/ts-1/wal/wals/70da09dabcf04b0c84fc877c804dd94d
I20260805 23:14:28.308583 12752 ts_tablet_manager.cc:1939] T 70da09dabcf04b0c84fc877c804dd94d P 0cce512cf8c94d27882b5a688f19e1b1: Deleting consensus metadata
I20260805 23:14:28.308687 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 12442
I20260805 23:14:28.309752 12727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:28.308975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.193:37505 (local address 127.0.0.1:36274)
0805 23:14:28.309563 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:28.309644 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.193:37505: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":469}
W20260805 23:14:28.309965 12226 catalog_manager.cc:4584] TS faad8cfd923345a48951165a95a200e1 (127.25.92.193:37505): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 70da09dabcf04b0c84fc877c804dd94d: Network error: Client connection negotiation failed: client connection to 127.25.92.193:37505: connect: Connection refused (error 111)
I20260805 23:14:28.315613 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 12582
I20260805 23:14:28.317059 12727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:28.315828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.194:45933 (local address 127.0.0.1:50984)
0805 23:14:28.316856 (+ 1028us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:28.316944 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.194:45933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":906}
I20260805 23:14:28.322824 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 12208
2026-08-05T23:14:28Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2438 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-05T23:14:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:14:28Z Disabled control of system clock
I20260805 23:14:28.355899 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971406251336-25971-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971406251336-25971-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971406251336-25971-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.254:43349
--webserver_interface=127.25.92.254
--webserver_port=0
--builtin_ntp_servers=127.25.92.212:41463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.92.254:43349
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260805 23:14:28.473276 12759 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:14:28.473615 12759 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:14:28.473655 12759 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:14:28.477764 12759 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:14:28.477838 12759 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:14:28.477854 12759 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:14:28.477871 12759 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:14:28.477887 12759 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:14:28.483384 12759 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:41463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971406251336-25971-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971406251336-25971-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.92.254:43349
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.254:43349
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.92.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12759
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971406251336-25971-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:14:28.484555 12759 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:14:28.485930 12759 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:14:28.495007 12766 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:14:28.495007 12768 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:14:28.495102 12765 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:14:28.495296 12759 server_base.cc:1034] running on GCE node
I20260805 23:14:28.495908 12759 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:14:28.497225 12759 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:14:28.498416 12759 hybrid_clock.cc:648] HybridClock initialized: now 1785971668498380 us; error 54 us; skew 500 ppm
I20260805 23:14:28.508057 12759 webserver.cc:460] Webserver started at http://127.25.92.254:42615/ using document root <none> and password file <none>
I20260805 23:14:28.508749 12759 fs_manager.cc:359] Metadata directory not provided
I20260805 23:14:28.508828 12759 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:14:28.509086 12759 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:14:28.511384 12759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971406251336-25971-0/minicluster-data/master-0/data/instance:
uuid: "43adec9805c44a508e87f1f9ad914e30"
format_stamp: "Formatted at 2026-08-05 23:14:28 on dist-test-slave-ss4n"
server_key: "ad43a69752dbf773118feb48ebb5a81e"
server_key_iv: "c76eabea9036c47fbad6270dc10b0500"
server_key_version: "encryptionkey@0"
I20260805 23:14:28.511978 12759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971406251336-25971-0/minicluster-data/master-0/wal/instance:
uuid: "43adec9805c44a508e87f1f9ad914e30"
format_stamp: "Formatted at 2026-08-05 23:14:28 on dist-test-slave-ss4n"
server_key: "ad43a69752dbf773118feb48ebb5a81e"
server_key_iv: "c76eabea9036c47fbad6270dc10b0500"
server_key_version: "encryptionkey@0"
I20260805 23:14:28.516136 12759 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260805 23:14:28.518918 12774 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:14:28.520344 12759 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260805 23:14:28.520509 12759 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971406251336-25971-0/minicluster-data/master-0/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971406251336-25971-0/minicluster-data/master-0/wal
uuid: "43adec9805c44a508e87f1f9ad914e30"
format_stamp: "Formatted at 2026-08-05 23:14:28 on dist-test-slave-ss4n"
server_key: "ad43a69752dbf773118feb48ebb5a81e"
server_key_iv: "c76eabea9036c47fbad6270dc10b0500"
server_key_version: "encryptionkey@0"
I20260805 23:14:28.520643 12759 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971406251336-25971-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971406251336-25971-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971406251336-25971-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:14:28.538147 12759 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:14:28.538932 12759 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:14:28.539160 12759 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:14:28.548038 12759 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.254:43349
I20260805 23:14:28.548074 12827 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.254:43349 every 8 connection(s)
I20260805 23:14:28.549170 12759 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
I20260805 23:14:28.552850 12828 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:14:28.553274 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 12759
I20260805 23:14:28.553435 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971406251336-25971-0/minicluster-data/master-0/wal/instance
I20260805 23:14:28.553798 25971 external_mini_cluster.cc:1428] Setting key 87698cbd78f1dd593ba5c162c19f8234
I20260805 23:14:28.559732 12828 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43adec9805c44a508e87f1f9ad914e30: Bootstrap starting.
I20260805 23:14:28.562613 12828 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 43adec9805c44a508e87f1f9ad914e30: Neither blocks nor log segments found. Creating new log.
I20260805 23:14:28.563495 12828 log.cc:824] T 00000000000000000000000000000000 P 43adec9805c44a508e87f1f9ad914e30: Log is configured to *not* fsync() on all Append() calls
I20260805 23:14:28.564311 12831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:28.555553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34996 (local address 127.25.92.254:43349)
0805 23:14:28.556280 (+ 727us) server_negotiation.cc:206] Beginning negotiation
0805 23:14:28.556296 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0805 23:14:28.556335 (+ 39us) server_negotiation.cc:407] Connection header received
0805 23:14:28.557008 (+ 673us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:14:28.557025 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:14:28.557443 (+ 418us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:14:28.557830 (+ 387us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:14:28.558208 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:28.559224 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:28.560486 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:28.560996 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:28.561148 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:28.562148 (+ 1000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:14:28.562153 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:14:28.562166 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:14:28.562634 (+ 468us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:14:28.562741 (+ 107us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:14:28.562774 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:14:28.562901 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0805 23:14:28.563069 (+ 168us) server_negotiation.cc:299] Negotiation successful
0805 23:14:28.563349 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":173,"threads_started":1}
I20260805 23:14:28.565960 12828 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43adec9805c44a508e87f1f9ad914e30: No bootstrap required, opened a new log
I20260805 23:14:28.570385 12828 raft_consensus.cc:348] T 00000000000000000000000000000000 P 43adec9805c44a508e87f1f9ad914e30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43adec9805c44a508e87f1f9ad914e30" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 43349 } }
I20260805 23:14:28.570737 12828 raft_consensus.cc:374] T 00000000000000000000000000000000 P 43adec9805c44a508e87f1f9ad914e30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:14:28.570830 12828 raft_consensus.cc:729] T 00000000000000000000000000000000 P 43adec9805c44a508e87f1f9ad914e30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43adec9805c44a508e87f1f9ad914e30, State: Initialized, Role: FOLLOWER
I20260805 23:14:28.571332 12828 consensus_queue.cc:260] T 00000000000000000000000000000000 P 43adec9805c44a508e87f1f9ad914e30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43adec9805c44a508e87f1f9ad914e30" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 43349 } }
I20260805 23:14:28.571489 12828 raft_consensus.cc:388] T 00000000000000000000000000000000 P 43adec9805c44a508e87f1f9ad914e30 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:14:28.571544 12828 raft_consensus.cc:482] T 00000000000000000000000000000000 P 43adec9805c44a508e87f1f9ad914e30 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:14:28.571627 12828 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 43adec9805c44a508e87f1f9ad914e30 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:14:28.572696 12828 raft_consensus.cc:504] T 00000000000000000000000000000000 P 43adec9805c44a508e87f1f9ad914e30 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43adec9805c44a508e87f1f9ad914e30" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 43349 } }
I20260805 23:14:28.573035 12828 leader_election.cc:302] T 00000000000000000000000000000000 P 43adec9805c44a508e87f1f9ad914e30 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 43adec9805c44a508e87f1f9ad914e30; no voters:
I20260805 23:14:28.573346 12828 leader_election.cc:288] T 00000000000000000000000000000000 P 43adec9805c44a508e87f1f9ad914e30 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 23:14:28.573562 12833 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 43adec9805c44a508e87f1f9ad914e30 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:14:28.573848 12833 raft_consensus.cc:686] T 00000000000000000000000000000000 P 43adec9805c44a508e87f1f9ad914e30 [term 1 LEADER]: Becoming Leader. State: Replica: 43adec9805c44a508e87f1f9ad914e30, State: Running, Role: LEADER
I20260805 23:14:28.574211 12833 consensus_queue.cc:237] T 00000000000000000000000000000000 P 43adec9805c44a508e87f1f9ad914e30 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43adec9805c44a508e87f1f9ad914e30" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 43349 } }
I20260805 23:14:28.574883 12828 sys_catalog.cc:564] T 00000000000000000000000000000000 P 43adec9805c44a508e87f1f9ad914e30 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:14:28.576042 12834 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43adec9805c44a508e87f1f9ad914e30 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "43adec9805c44a508e87f1f9ad914e30" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43adec9805c44a508e87f1f9ad914e30" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 43349 } } }
I20260805 23:14:28.576242 12834 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43adec9805c44a508e87f1f9ad914e30 [sys.catalog]: This master's current role is: LEADER
I20260805 23:14:28.576640 12842 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:14:28.576592 12835 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43adec9805c44a508e87f1f9ad914e30 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 43adec9805c44a508e87f1f9ad914e30. Latest consensus state: current_term: 1 leader_uuid: "43adec9805c44a508e87f1f9ad914e30" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43adec9805c44a508e87f1f9ad914e30" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 43349 } } }
I20260805 23:14:28.577368 12835 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43adec9805c44a508e87f1f9ad914e30 [sys.catalog]: This master's current role is: LEADER
I20260805 23:14:28.580304 12842 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:14:28.587610 12842 catalog_manager.cc:1347] Generated new cluster ID: 7dfe5c23d0b845d797bc73b82f722ef5
I20260805 23:14:28.587735 12842 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:14:28.601333 12842 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:14:28.602730 12842 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:14:28.608083 12842 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 43adec9805c44a508e87f1f9ad914e30: Generated new TSK 0
I20260805 23:14:28.608953 12842 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260805 23:14:28.616437 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971406251336-25971-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971406251336-25971-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971406251336-25971-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971406251336-25971-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.92.193:0
--local_ip_for_outbound_sockets=127.25.92.193
--webserver_interface=127.25.92.193
--webserver_port=0
--tserver_master_addrs=127.25.92.254:43349
--builtin_ntp_servers=127.25.92.212:41463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260805 23:14:28.736789 12852 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:14:28.737080 12852 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:14:28.737123 12852 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:14:28.741313 12852 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:14:28.741398 12852 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:14:28.741457 12852 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.92.193
I20260805 23:14:28.746575 12852 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:41463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971406251336-25971-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971406251336-25971-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.92.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971406251336-25971-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.92.193
--webserver_port=0
--tserver_master_addrs=127.25.92.254:43349
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12852
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.92.193
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971406251336-25971-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:14:28.747864 12852 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:14:28.749408 12852 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:14:28.759706 12860 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:14:28.759706 12858 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:14:28.759697 12857 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:14:28.760074 12852 server_base.cc:1034] running on GCE node
I20260805 23:14:28.760639 12852 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:14:28.761376 12852 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:14:28.762709 12852 hybrid_clock.cc:648] HybridClock initialized: now 1785971668762673 us; error 55 us; skew 500 ppm
I20260805 23:14:28.765177 12852 webserver.cc:460] Webserver started at http://127.25.92.193:42331/ using document root <none> and password file <none>
I20260805 23:14:28.765969 12852 fs_manager.cc:359] Metadata directory not provided
I20260805 23:14:28.766083 12852 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:14:28.766350 12852 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:14:28.768915 12852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971406251336-25971-0/minicluster-data/ts-0/data/instance:
uuid: "12ae1908f90a4ce3a9c7a2dd5a13a170"
format_stamp: "Formatted at 2026-08-05 23:14:28 on dist-test-slave-ss4n"
server_key: "d0c0194e37473772c518f90734f90bd4"
server_key_iv: "c7a8821021a678da7f670eef5da5edab"
server_key_version: "encryptionkey@0"
I20260805 23:14:28.769680 12852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971406251336-25971-0/minicluster-data/ts-0/wal/instance:
uuid: "12ae1908f90a4ce3a9c7a2dd5a13a170"
format_stamp: "Formatted at 2026-08-05 23:14:28 on dist-test-slave-ss4n"
server_key: "d0c0194e37473772c518f90734f90bd4"
server_key_iv: "c7a8821021a678da7f670eef5da5edab"
server_key_version: "encryptionkey@0"
I20260805 23:14:28.774302 12852 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260805 23:14:28.777297 12866 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:14:28.779094 12852 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:14:28.779297 12852 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971406251336-25971-0/minicluster-data/ts-0/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971406251336-25971-0/minicluster-data/ts-0/wal
uuid: "12ae1908f90a4ce3a9c7a2dd5a13a170"
format_stamp: "Formatted at 2026-08-05 23:14:28 on dist-test-slave-ss4n"
server_key: "d0c0194e37473772c518f90734f90bd4"
server_key_iv: "c7a8821021a678da7f670eef5da5edab"
server_key_version: "encryptionkey@0"
I20260805 23:14:28.779449 12852 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971406251336-25971-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971406251336-25971-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971406251336-25971-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:14:28.802455 12852 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:14:28.803364 12852 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:14:28.803637 12852 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:14:28.804390 12852 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:14:28.805626 12852 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:14:28.805718 12852 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:14:28.805811 12852 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:14:28.805866 12852 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:14:28.817490 12852 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.193:44073
I20260805 23:14:28.817541 12980 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.193:44073 every 8 connection(s)
I20260805 23:14:28.818560 12852 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971406251336-25971-0/minicluster-data/ts-0/data/info.pb
I20260805 23:14:28.824600 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 12852
I20260805 23:14:28.824777 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971406251336-25971-0/minicluster-data/ts-0/wal/instance
I20260805 23:14:28.825132 25971 external_mini_cluster.cc:1428] Setting key faea33641d6d1d58ef32d32d1ed321fe
I20260805 23:14:28.828016 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971406251336-25971-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971406251336-25971-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971406251336-25971-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971406251336-25971-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.92.194:0
--local_ip_for_outbound_sockets=127.25.92.194
--webserver_interface=127.25.92.194
--webserver_port=0
--tserver_master_addrs=127.25.92.254:43349
--builtin_ntp_servers=127.25.92.212:41463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260805 23:14:28.831532 12831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:28.820658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.193:36313 (local address 127.25.92.254:43349)
0805 23:14:28.820968 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0805 23:14:28.820973 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:14:28.822169 (+ 1196us) server_negotiation.cc:407] Connection header received
0805 23:14:28.823863 (+ 1694us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:14:28.823871 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:14:28.823985 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:14:28.824179 (+ 194us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:14:28.825996 (+ 1817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:28.826634 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:28.828618 (+ 1984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:28.828874 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:28.828981 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:28.830792 (+ 1811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:14:28.830799 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:14:28.830806 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:14:28.830828 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:14:28.830857 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:14:28.830872 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:14:28.830966 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0805 23:14:28.831197 (+ 231us) server_negotiation.cc:299] Negotiation successful
0805 23:14:28.831301 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129}
I20260805 23:14:28.832445 12983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:28.821089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:43349 (local address 127.25.92.193:36313)
0805 23:14:28.821965 (+ 876us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:28.822013 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0805 23:14:28.823414 (+ 1401us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:14:28.824540 (+ 1126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:14:28.824553 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:14:28.824701 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:14:28.825779 (+ 1078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:14:28.825800 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:14:28.826820 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:14:28.826824 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:14:28.828393 (+ 1569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:14:28.828412 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:14:28.829097 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:14:28.829105 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:14:28.829405 (+ 300us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:28.830398 (+ 993us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0805 23:14:28.830400 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:14:28.830464 (+ 64us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0805 23:14:28.830568 (+ 104us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:14:28.830991 (+ 423us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:14:28.831000 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:14:28.831007 (+ 7us) client_negotiation.cc:769] Sending connection context
0805 23:14:28.831154 (+ 147us) client_negotiation.cc:240] Negotiation successful
0805 23:14:28.831497 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":145,"threads_started":1}
I20260805 23:14:28.833925 12981 heartbeater.cc:347] Connected to a master server at 127.25.92.254:43349
I20260805 23:14:28.834236 12981 heartbeater.cc:464] Registering TS with master...
I20260805 23:14:28.835024 12981 heartbeater.cc:511] Master 127.25.92.254:43349 requested a full tablet report, sending...
I20260805 23:14:28.837205 12791 ts_manager.cc:194] Registered new tserver with Master: 12ae1908f90a4ce3a9c7a2dd5a13a170 (127.25.92.193:44073)
I20260805 23:14:28.839134 12791 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.25.92.193:36313
W20260805 23:14:28.947543 12984 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:14:28.947861 12984 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:14:28.947902 12984 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:14:28.952723 12984 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:14:28.952812 12984 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:14:28.952874 12984 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.92.194
I20260805 23:14:28.958433 12984 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:41463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971406251336-25971-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971406251336-25971-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.92.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971406251336-25971-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.92.194
--webserver_port=0
--tserver_master_addrs=127.25.92.254:43349
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12984
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.92.194
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971406251336-25971-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:14:28.959537 12984 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:14:28.960947 12984 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:14:28.971549 12990 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:14:28.971549 12992 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:14:28.971799 12989 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:14:28.972012 12984 server_base.cc:1034] running on GCE node
I20260805 23:14:28.972434 12984 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:14:28.973081 12984 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:14:28.974267 12984 hybrid_clock.cc:648] HybridClock initialized: now 1785971668974244 us; error 44 us; skew 500 ppm
I20260805 23:14:28.976608 12984 webserver.cc:460] Webserver started at http://127.25.92.194:41631/ using document root <none> and password file <none>
I20260805 23:14:28.977332 12984 fs_manager.cc:359] Metadata directory not provided
I20260805 23:14:28.977403 12984 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:14:28.977667 12984 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:14:28.979894 12984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971406251336-25971-0/minicluster-data/ts-1/data/instance:
uuid: "4ed3ccdab406486cb73b637d14de8fb7"
format_stamp: "Formatted at 2026-08-05 23:14:28 on dist-test-slave-ss4n"
server_key: "b429ef283a7cc70737386e31ecebd2b0"
server_key_iv: "4a88d5e4b5bf35b4296acea3533d26b2"
server_key_version: "encryptionkey@0"
I20260805 23:14:28.980523 12984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971406251336-25971-0/minicluster-data/ts-1/wal/instance:
uuid: "4ed3ccdab406486cb73b637d14de8fb7"
format_stamp: "Formatted at 2026-08-05 23:14:28 on dist-test-slave-ss4n"
server_key: "b429ef283a7cc70737386e31ecebd2b0"
server_key_iv: "4a88d5e4b5bf35b4296acea3533d26b2"
server_key_version: "encryptionkey@0"
I20260805 23:14:28.984808 12984 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260805 23:14:28.987630 12998 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:14:28.989008 12984 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260805 23:14:28.989171 12984 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971406251336-25971-0/minicluster-data/ts-1/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971406251336-25971-0/minicluster-data/ts-1/wal
uuid: "4ed3ccdab406486cb73b637d14de8fb7"
format_stamp: "Formatted at 2026-08-05 23:14:28 on dist-test-slave-ss4n"
server_key: "b429ef283a7cc70737386e31ecebd2b0"
server_key_iv: "4a88d5e4b5bf35b4296acea3533d26b2"
server_key_version: "encryptionkey@0"
I20260805 23:14:28.989300 12984 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971406251336-25971-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971406251336-25971-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971406251336-25971-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:14:29.005537 12984 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:14:29.006359 12984 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:14:29.006577 12984 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:14:29.007221 12984 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:14:29.008246 12984 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:14:29.008296 12984 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:14:29.008337 12984 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:14:29.008391 12984 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:14:29.018379 12984 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.194:41817
I20260805 23:14:29.018404 13114 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.194:41817 every 8 connection(s)
I20260805 23:14:29.019404 12984 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971406251336-25971-0/minicluster-data/ts-1/data/info.pb
I20260805 23:14:29.027093 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 12984
I20260805 23:14:29.027262 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971406251336-25971-0/minicluster-data/ts-1/wal/instance
I20260805 23:14:29.027603 25971 external_mini_cluster.cc:1428] Setting key 9e03c5021056ed2d1d12441bc6c1f89a
I20260805 23:14:29.030252 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971406251336-25971-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971406251336-25971-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971406251336-25971-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971406251336-25971-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.92.195:0
--local_ip_for_outbound_sockets=127.25.92.195
--webserver_interface=127.25.92.195
--webserver_port=0
--tserver_master_addrs=127.25.92.254:43349
--builtin_ntp_servers=127.25.92.212:41463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260805 23:14:29.031158 12831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:29.021602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.194:54705 (local address 127.25.92.254:43349)
0805 23:14:29.021760 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0805 23:14:29.021764 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:14:29.022968 (+ 1204us) server_negotiation.cc:407] Connection header received
0805 23:14:29.024633 (+ 1665us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:14:29.024636 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:14:29.024751 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:14:29.024868 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:14:29.025999 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:29.026543 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:29.028559 (+ 2016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:29.028862 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:29.028978 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:29.030606 (+ 1628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:14:29.030609 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:14:29.030611 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:14:29.030626 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:14:29.030646 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:14:29.030655 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:14:29.030721 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0805 23:14:29.030952 (+ 231us) server_negotiation.cc:299] Negotiation successful
0805 23:14:29.031016 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260805 23:14:29.032477 13117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:29.022021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:43349 (local address 127.25.92.194:54705)
0805 23:14:29.022785 (+ 764us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:29.022833 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0805 23:14:29.024399 (+ 1566us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:14:29.025093 (+ 694us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:14:29.025101 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:14:29.025200 (+ 99us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:14:29.025823 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:14:29.025839 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:14:29.026752 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:14:29.026757 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:14:29.028335 (+ 1578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:14:29.028351 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:14:29.029069 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:14:29.029075 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:14:29.029334 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:29.030183 (+ 849us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0805 23:14:29.030185 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:14:29.030256 (+ 71us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0805 23:14:29.030380 (+ 124us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:14:29.030757 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:14:29.030772 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:14:29.030783 (+ 11us) client_negotiation.cc:769] Sending connection context
0805 23:14:29.030934 (+ 151us) client_negotiation.cc:240] Negotiation successful
0805 23:14:29.031326 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":162,"threads_started":1}
I20260805 23:14:29.034139 13115 heartbeater.cc:347] Connected to a master server at 127.25.92.254:43349
I20260805 23:14:29.034464 13115 heartbeater.cc:464] Registering TS with master...
I20260805 23:14:29.035128 13115 heartbeater.cc:511] Master 127.25.92.254:43349 requested a full tablet report, sending...
I20260805 23:14:29.036463 12791 ts_manager.cc:194] Registered new tserver with Master: 4ed3ccdab406486cb73b637d14de8fb7 (127.25.92.194:41817)
I20260805 23:14:29.037282 12791 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.25.92.194:54705
W20260805 23:14:29.153409 13118 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:14:29.153810 13118 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:14:29.153887 13118 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:14:29.158208 13118 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:14:29.158293 13118 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:14:29.158390 13118 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.92.195
I20260805 23:14:29.163548 13118 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:41463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971406251336-25971-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971406251336-25971-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.92.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971406251336-25971-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.92.195
--webserver_port=0
--tserver_master_addrs=127.25.92.254:43349
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13118
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.92.195
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971406251336-25971-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:14:29.164747 13118 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:14:29.166296 13118 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:14:29.176630 13124 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:14:29.176640 13127 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:14:29.176679 13118 server_base.cc:1034] running on GCE node
W20260805 23:14:29.176944 13125 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:14:29.177456 13118 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:14:29.178257 13118 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:14:29.179432 13118 hybrid_clock.cc:648] HybridClock initialized: now 1785971669179415 us; error 40 us; skew 500 ppm
I20260805 23:14:29.181710 13118 webserver.cc:460] Webserver started at http://127.25.92.195:43651/ using document root <none> and password file <none>
I20260805 23:14:29.182399 13118 fs_manager.cc:359] Metadata directory not provided
I20260805 23:14:29.182495 13118 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:14:29.182798 13118 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:14:29.185372 13118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971406251336-25971-0/minicluster-data/ts-2/data/instance:
uuid: "1b477a4ce180447cac00168c49fd41d7"
format_stamp: "Formatted at 2026-08-05 23:14:29 on dist-test-slave-ss4n"
server_key: "a382fb70ca153ac33cf1c697f37cca06"
server_key_iv: "1d7fd8aec6f833618aa1bc3e362f3d92"
server_key_version: "encryptionkey@0"
I20260805 23:14:29.186096 13118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971406251336-25971-0/minicluster-data/ts-2/wal/instance:
uuid: "1b477a4ce180447cac00168c49fd41d7"
format_stamp: "Formatted at 2026-08-05 23:14:29 on dist-test-slave-ss4n"
server_key: "a382fb70ca153ac33cf1c697f37cca06"
server_key_iv: "1d7fd8aec6f833618aa1bc3e362f3d92"
server_key_version: "encryptionkey@0"
I20260805 23:14:29.190347 13118 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260805 23:14:29.193161 13133 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:14:29.194741 13118 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260805 23:14:29.194871 13118 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971406251336-25971-0/minicluster-data/ts-2/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971406251336-25971-0/minicluster-data/ts-2/wal
uuid: "1b477a4ce180447cac00168c49fd41d7"
format_stamp: "Formatted at 2026-08-05 23:14:29 on dist-test-slave-ss4n"
server_key: "a382fb70ca153ac33cf1c697f37cca06"
server_key_iv: "1d7fd8aec6f833618aa1bc3e362f3d92"
server_key_version: "encryptionkey@0"
I20260805 23:14:29.194999 13118 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971406251336-25971-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971406251336-25971-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971406251336-25971-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:14:29.215623 13118 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:14:29.216353 13118 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:14:29.216516 13118 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:14:29.217119 13118 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:14:29.218122 13118 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:14:29.218171 13118 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:14:29.218212 13118 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:14:29.218227 13118 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:14:29.227955 13247 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.195:46275 every 8 connection(s)
I20260805 23:14:29.227952 13118 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.195:46275
I20260805 23:14:29.229271 13118 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971406251336-25971-0/minicluster-data/ts-2/data/info.pb
I20260805 23:14:29.238970 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 13118
I20260805 23:14:29.239120 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971406251336-25971-0/minicluster-data/ts-2/wal/instance
I20260805 23:14:29.239234 12831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:29.231336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.195:53335 (local address 127.25.92.254:43349)
0805 23:14:29.231511 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0805 23:14:29.231516 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:14:29.232810 (+ 1294us) server_negotiation.cc:407] Connection header received
0805 23:14:29.234230 (+ 1420us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:14:29.234232 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:14:29.234287 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:14:29.234370 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:14:29.235331 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:29.235868 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:29.237061 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:29.237283 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:29.237392 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:29.238630 (+ 1238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:14:29.238633 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:14:29.238635 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:14:29.238650 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:14:29.238670 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:14:29.238678 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:14:29.238739 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0805 23:14:29.238948 (+ 209us) server_negotiation.cc:299] Negotiation successful
0805 23:14:29.239016 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260805 23:14:29.239421 25971 external_mini_cluster.cc:1428] Setting key 89a8d15ae03f10e916dbecbdd956e02c
I20260805 23:14:29.240314 13250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:29.231796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:43349 (local address 127.25.92.195:53335)
0805 23:14:29.232494 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:29.232555 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0805 23:14:29.233984 (+ 1429us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:14:29.234549 (+ 565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:14:29.234556 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:14:29.234681 (+ 125us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:14:29.235191 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:14:29.235204 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:14:29.236023 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:14:29.236026 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:14:29.236943 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:14:29.236953 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:14:29.237474 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:14:29.237477 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:14:29.237667 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:29.238302 (+ 635us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0805 23:14:29.238304 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:14:29.238348 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0805 23:14:29.238420 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:14:29.238768 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:14:29.238781 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:14:29.238790 (+ 9us) client_negotiation.cc:769] Sending connection context
0805 23:14:29.238942 (+ 152us) client_negotiation.cc:240] Negotiation successful
0805 23:14:29.239280 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":123,"threads_started":1}
I20260805 23:14:29.241648 13248 heartbeater.cc:347] Connected to a master server at 127.25.92.254:43349
I20260805 23:14:29.241932 13248 heartbeater.cc:464] Registering TS with master...
I20260805 23:14:29.242528 13248 heartbeater.cc:511] Master 127.25.92.254:43349 requested a full tablet report, sending...
I20260805 23:14:29.243661 12791 ts_manager.cc:194] Registered new tserver with Master: 1b477a4ce180447cac00168c49fd41d7 (127.25.92.195:46275)
I20260805 23:14:29.244225 12791 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.25.92.195:53335
I20260805 23:14:29.254156 25971 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260805 23:14:29.260605 12831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:29.256532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35004 (local address 127.25.92.254:43349)
0805 23:14:29.256681 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0805 23:14:29.256685 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:14:29.256933 (+ 248us) server_negotiation.cc:407] Connection header received
0805 23:14:29.257010 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:14:29.257012 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:14:29.257062 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:14:29.257135 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:14:29.257436 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:29.258028 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:29.258935 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:29.259147 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:29.259240 (+ 93us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:29.260130 (+ 890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:14:29.260134 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:14:29.260136 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:14:29.260162 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:14:29.260182 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:14:29.260190 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:14:29.260294 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0805 23:14:29.260440 (+ 146us) server_negotiation.cc:299] Negotiation successful
0805 23:14:29.260478 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260805 23:14:29.264159 12791 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35004:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260805 23:14:29.267555 12791 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260805 23:14:29.284308 13260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:29.277073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56852 (local address 127.25.92.195:46275)
0805 23:14:29.279481 (+ 2408us) server_negotiation.cc:206] Beginning negotiation
0805 23:14:29.279492 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0805 23:14:29.279515 (+ 23us) server_negotiation.cc:407] Connection header received
0805 23:14:29.279586 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:14:29.279590 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:14:29.279775 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:14:29.279921 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:14:29.280172 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:29.281194 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:29.282522 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:29.283425 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:29.283568 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:29.283857 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0805 23:14:29.284010 (+ 153us) server_negotiation.cc:299] Negotiation successful
0805 23:14:29.284116 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2292,"thread_start_us":99,"threads_started":1}
I20260805 23:14:29.284415 13262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:29.277109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.193:44073 (local address 127.0.0.1:59718)
0805 23:14:29.278397 (+ 1288us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:29.278411 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0805 23:14:29.278515 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:14:29.279031 (+ 516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:14:29.279044 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:14:29.279064 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:14:29.279285 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:14:29.279294 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:14:29.281301 (+ 2007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:14:29.281305 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:14:29.282506 (+ 1201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:14:29.282515 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:14:29.283885 (+ 1370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:14:29.283890 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:14:29.284018 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:29.284038 (+ 20us) client_negotiation.cc:769] Sending connection context
0805 23:14:29.284147 (+ 109us) client_negotiation.cc:240] Negotiation successful
0805 23:14:29.284231 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1223,"mutex_wait_us":42,"thread_start_us":109,"threads_started":1}
I20260805 23:14:29.284631 13261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:29.277527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.194:41817 (local address 127.0.0.1:48704)
0805 23:14:29.278515 (+ 988us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:29.278530 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0805 23:14:29.278629 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:14:29.279179 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:14:29.279182 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:14:29.279196 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:14:29.279303 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:14:29.279308 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:14:29.280709 (+ 1401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:14:29.280713 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:14:29.282217 (+ 1504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:14:29.282229 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:14:29.283551 (+ 1322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:14:29.283555 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:14:29.284291 (+ 736us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:29.284318 (+ 27us) client_negotiation.cc:769] Sending connection context
0805 23:14:29.284404 (+ 86us) client_negotiation.cc:240] Negotiation successful
0805 23:14:29.284497 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":916,"mutex_wait_us":582,"thread_start_us":118,"threads_started":1}
I20260805 23:14:29.284914 13263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:29.277753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59718 (local address 127.25.92.193:44073)
0805 23:14:29.278152 (+ 399us) server_negotiation.cc:206] Beginning negotiation
0805 23:14:29.278160 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0805 23:14:29.278520 (+ 360us) server_negotiation.cc:407] Connection header received
0805 23:14:29.278642 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:14:29.278647 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:14:29.278869 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:14:29.279030 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:14:29.280086 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:29.281162 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:29.282632 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:29.283530 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:29.283636 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:29.283808 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0805 23:14:29.284621 (+ 813us) server_negotiation.cc:299] Negotiation successful
0805 23:14:29.284740 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":82,"threads_started":1}
I20260805 23:14:29.284960 13264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:29.277964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48704 (local address 127.25.92.194:41817)
0805 23:14:29.278774 (+ 810us) server_negotiation.cc:206] Beginning negotiation
0805 23:14:29.278786 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0805 23:14:29.278807 (+ 21us) server_negotiation.cc:407] Connection header received
0805 23:14:29.278876 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:14:29.278882 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:14:29.279044 (+ 162us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:14:29.279181 (+ 137us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:14:29.279417 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:29.280570 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:29.282377 (+ 1807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:29.283204 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:29.283274 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:29.283431 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0805 23:14:29.284580 (+ 1149us) server_negotiation.cc:299] Negotiation successful
0805 23:14:29.284775 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":710,"thread_start_us":149,"threads_started":1}
I20260805 23:14:29.285102 13259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:29.276956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.195:46275 (local address 127.0.0.1:56852)
0805 23:14:29.277863 (+ 907us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:29.277921 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0805 23:14:29.278056 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:14:29.279918 (+ 1862us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:14:29.279920 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:14:29.279937 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:14:29.280008 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:14:29.280013 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:14:29.281342 (+ 1329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:14:29.281349 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:14:29.282375 (+ 1026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:14:29.282386 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:14:29.283557 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:14:29.283560 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:14:29.283771 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:29.283818 (+ 47us) client_negotiation.cc:769] Sending connection context
0805 23:14:29.283967 (+ 149us) client_negotiation.cc:240] Negotiation successful
0805 23:14:29.284112 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":729,"mutex_wait_us":47,"thread_start_us":100,"threads_started":1}
I20260805 23:14:29.288045 13048 tablet_service.cc:1467] Processing CreateTablet for tablet b75c37e7c3ed440c9523223d376c9979 (DEFAULT_TABLE table=test-table [id=6facdbfdf5094b85854dbf7e49b762af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:14:29.288043 12915 tablet_service.cc:1467] Processing CreateTablet for tablet b75c37e7c3ed440c9523223d376c9979 (DEFAULT_TABLE table=test-table [id=6facdbfdf5094b85854dbf7e49b762af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:14:29.289219 13048 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b75c37e7c3ed440c9523223d376c9979. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:14:29.289251 12915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b75c37e7c3ed440c9523223d376c9979. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:14:29.292063 13181 tablet_service.cc:1467] Processing CreateTablet for tablet b75c37e7c3ed440c9523223d376c9979 (DEFAULT_TABLE table=test-table [id=6facdbfdf5094b85854dbf7e49b762af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:14:29.293242 13181 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b75c37e7c3ed440c9523223d376c9979. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:14:29.296545 13265 tablet_bootstrap.cc:492] T b75c37e7c3ed440c9523223d376c9979 P 12ae1908f90a4ce3a9c7a2dd5a13a170: Bootstrap starting.
I20260805 23:14:29.299165 13265 tablet_bootstrap.cc:654] T b75c37e7c3ed440c9523223d376c9979 P 12ae1908f90a4ce3a9c7a2dd5a13a170: Neither blocks nor log segments found. Creating new log.
I20260805 23:14:29.300194 13265 log.cc:824] T b75c37e7c3ed440c9523223d376c9979 P 12ae1908f90a4ce3a9c7a2dd5a13a170: Log is configured to *not* fsync() on all Append() calls
I20260805 23:14:29.300228 13266 tablet_bootstrap.cc:492] T b75c37e7c3ed440c9523223d376c9979 P 1b477a4ce180447cac00168c49fd41d7: Bootstrap starting.
I20260805 23:14:29.303371 13266 tablet_bootstrap.cc:654] T b75c37e7c3ed440c9523223d376c9979 P 1b477a4ce180447cac00168c49fd41d7: Neither blocks nor log segments found. Creating new log.
I20260805 23:14:29.304301 13266 log.cc:824] T b75c37e7c3ed440c9523223d376c9979 P 1b477a4ce180447cac00168c49fd41d7: Log is configured to *not* fsync() on all Append() calls
I20260805 23:14:29.305804 13265 tablet_bootstrap.cc:492] T b75c37e7c3ed440c9523223d376c9979 P 12ae1908f90a4ce3a9c7a2dd5a13a170: No bootstrap required, opened a new log
I20260805 23:14:29.306057 13265 ts_tablet_manager.cc:1397] T b75c37e7c3ed440c9523223d376c9979 P 12ae1908f90a4ce3a9c7a2dd5a13a170: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.004s
I20260805 23:14:29.306715 13266 tablet_bootstrap.cc:492] T b75c37e7c3ed440c9523223d376c9979 P 1b477a4ce180447cac00168c49fd41d7: No bootstrap required, opened a new log
I20260805 23:14:29.306915 13266 ts_tablet_manager.cc:1397] T b75c37e7c3ed440c9523223d376c9979 P 1b477a4ce180447cac00168c49fd41d7: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260805 23:14:29.307245 13269 tablet_bootstrap.cc:492] T b75c37e7c3ed440c9523223d376c9979 P 4ed3ccdab406486cb73b637d14de8fb7: Bootstrap starting.
I20260805 23:14:29.310006 13269 tablet_bootstrap.cc:654] T b75c37e7c3ed440c9523223d376c9979 P 4ed3ccdab406486cb73b637d14de8fb7: Neither blocks nor log segments found. Creating new log.
I20260805 23:14:29.310817 13269 log.cc:824] T b75c37e7c3ed440c9523223d376c9979 P 4ed3ccdab406486cb73b637d14de8fb7: Log is configured to *not* fsync() on all Append() calls
I20260805 23:14:29.311998 13265 raft_consensus.cc:348] T b75c37e7c3ed440c9523223d376c9979 P 12ae1908f90a4ce3a9c7a2dd5a13a170 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b477a4ce180447cac00168c49fd41d7" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 46275 } } peers { permanent_uuid: "12ae1908f90a4ce3a9c7a2dd5a13a170" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 44073 } } peers { permanent_uuid: "4ed3ccdab406486cb73b637d14de8fb7" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 41817 } }
I20260805 23:14:29.312366 13265 raft_consensus.cc:374] T b75c37e7c3ed440c9523223d376c9979 P 12ae1908f90a4ce3a9c7a2dd5a13a170 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:14:29.312436 13265 raft_consensus.cc:729] T b75c37e7c3ed440c9523223d376c9979 P 12ae1908f90a4ce3a9c7a2dd5a13a170 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12ae1908f90a4ce3a9c7a2dd5a13a170, State: Initialized, Role: FOLLOWER
I20260805 23:14:29.312458 13266 raft_consensus.cc:348] T b75c37e7c3ed440c9523223d376c9979 P 1b477a4ce180447cac00168c49fd41d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b477a4ce180447cac00168c49fd41d7" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 46275 } } peers { permanent_uuid: "12ae1908f90a4ce3a9c7a2dd5a13a170" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 44073 } } peers { permanent_uuid: "4ed3ccdab406486cb73b637d14de8fb7" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 41817 } }
I20260805 23:14:29.313227 13266 raft_consensus.cc:374] T b75c37e7c3ed440c9523223d376c9979 P 1b477a4ce180447cac00168c49fd41d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:14:29.313297 13266 raft_consensus.cc:729] T b75c37e7c3ed440c9523223d376c9979 P 1b477a4ce180447cac00168c49fd41d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b477a4ce180447cac00168c49fd41d7, State: Initialized, Role: FOLLOWER
I20260805 23:14:29.313452 13265 consensus_queue.cc:260] T b75c37e7c3ed440c9523223d376c9979 P 12ae1908f90a4ce3a9c7a2dd5a13a170 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b477a4ce180447cac00168c49fd41d7" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 46275 } } peers { permanent_uuid: "12ae1908f90a4ce3a9c7a2dd5a13a170" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 44073 } } peers { permanent_uuid: "4ed3ccdab406486cb73b637d14de8fb7" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 41817 } }
I20260805 23:14:29.313774 13266 consensus_queue.cc:260] T b75c37e7c3ed440c9523223d376c9979 P 1b477a4ce180447cac00168c49fd41d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b477a4ce180447cac00168c49fd41d7" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 46275 } } peers { permanent_uuid: "12ae1908f90a4ce3a9c7a2dd5a13a170" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 44073 } } peers { permanent_uuid: "4ed3ccdab406486cb73b637d14de8fb7" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 41817 } }
I20260805 23:14:29.313912 13269 tablet_bootstrap.cc:492] T b75c37e7c3ed440c9523223d376c9979 P 4ed3ccdab406486cb73b637d14de8fb7: No bootstrap required, opened a new log
I20260805 23:14:29.314152 13269 ts_tablet_manager.cc:1397] T b75c37e7c3ed440c9523223d376c9979 P 4ed3ccdab406486cb73b637d14de8fb7: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.002s
I20260805 23:14:29.314519 12981 heartbeater.cc:503] Master 127.25.92.254:43349 was elected leader, sending a full tablet report...
I20260805 23:14:29.314817 13265 ts_tablet_manager.cc:1428] T b75c37e7c3ed440c9523223d376c9979 P 12ae1908f90a4ce3a9c7a2dd5a13a170: Time spent starting tablet: real 0.009s user 0.008s sys 0.000s
I20260805 23:14:29.314817 13266 ts_tablet_manager.cc:1428] T b75c37e7c3ed440c9523223d376c9979 P 1b477a4ce180447cac00168c49fd41d7: Time spent starting tablet: real 0.008s user 0.008s sys 0.000s
I20260805 23:14:29.315204 13248 heartbeater.cc:503] Master 127.25.92.254:43349 was elected leader, sending a full tablet report...
I20260805 23:14:29.319597 13269 raft_consensus.cc:348] T b75c37e7c3ed440c9523223d376c9979 P 4ed3ccdab406486cb73b637d14de8fb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b477a4ce180447cac00168c49fd41d7" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 46275 } } peers { permanent_uuid: "12ae1908f90a4ce3a9c7a2dd5a13a170" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 44073 } } peers { permanent_uuid: "4ed3ccdab406486cb73b637d14de8fb7" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 41817 } }
W20260805 23:14:29.319937 12982 tablet.cc:2369] T b75c37e7c3ed440c9523223d376c9979 P 12ae1908f90a4ce3a9c7a2dd5a13a170: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260805 23:14:29.320004 13269 raft_consensus.cc:374] T b75c37e7c3ed440c9523223d376c9979 P 4ed3ccdab406486cb73b637d14de8fb7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:14:29.320091 13269 raft_consensus.cc:729] T b75c37e7c3ed440c9523223d376c9979 P 4ed3ccdab406486cb73b637d14de8fb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ed3ccdab406486cb73b637d14de8fb7, State: Initialized, Role: FOLLOWER
I20260805 23:14:29.320609 13269 consensus_queue.cc:260] T b75c37e7c3ed440c9523223d376c9979 P 4ed3ccdab406486cb73b637d14de8fb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b477a4ce180447cac00168c49fd41d7" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 46275 } } peers { permanent_uuid: "12ae1908f90a4ce3a9c7a2dd5a13a170" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 44073 } } peers { permanent_uuid: "4ed3ccdab406486cb73b637d14de8fb7" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 41817 } }
I20260805 23:14:29.321385 13115 heartbeater.cc:503] Master 127.25.92.254:43349 was elected leader, sending a full tablet report...
I20260805 23:14:29.321888 13269 ts_tablet_manager.cc:1428] T b75c37e7c3ed440c9523223d376c9979 P 4ed3ccdab406486cb73b637d14de8fb7: Time spent starting tablet: real 0.008s user 0.005s sys 0.002s
W20260805 23:14:29.480360 13249 tablet.cc:2369] T b75c37e7c3ed440c9523223d376c9979 P 1b477a4ce180447cac00168c49fd41d7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260805 23:14:29.512715 13272 raft_consensus.cc:482] T b75c37e7c3ed440c9523223d376c9979 P 1b477a4ce180447cac00168c49fd41d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 23:14:29.512955 13272 raft_consensus.cc:504] T b75c37e7c3ed440c9523223d376c9979 P 1b477a4ce180447cac00168c49fd41d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b477a4ce180447cac00168c49fd41d7" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 46275 } } peers { permanent_uuid: "12ae1908f90a4ce3a9c7a2dd5a13a170" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 44073 } } peers { permanent_uuid: "4ed3ccdab406486cb73b637d14de8fb7" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 41817 } }
I20260805 23:14:29.514303 13272 leader_election.cc:288] T b75c37e7c3ed440c9523223d376c9979 P 1b477a4ce180447cac00168c49fd41d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12ae1908f90a4ce3a9c7a2dd5a13a170 (127.25.92.193:44073), 4ed3ccdab406486cb73b637d14de8fb7 (127.25.92.194:41817)
I20260805 23:14:29.518499 13263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:29.514753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.195:53789 (local address 127.25.92.193:44073)
0805 23:14:29.514887 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0805 23:14:29.514892 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:14:29.515064 (+ 172us) server_negotiation.cc:407] Connection header received
0805 23:14:29.515147 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:14:29.515153 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:14:29.515222 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:14:29.515332 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:14:29.515728 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:29.516376 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:29.517459 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:29.517903 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:29.518045 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:29.518098 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0805 23:14:29.518297 (+ 199us) server_negotiation.cc:299] Negotiation successful
0805 23:14:29.518378 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260805 23:14:29.518498 13250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:29.514611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.193:44073 (local address 127.25.92.195:53789)
0805 23:14:29.514868 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:29.514885 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0805 23:14:29.515012 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:14:29.515394 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:14:29.515398 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:14:29.515422 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:14:29.515563 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:14:29.515571 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:14:29.516497 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:14:29.516502 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:14:29.517321 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:14:29.517331 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:14:29.518039 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:14:29.518043 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:14:29.518154 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:29.518178 (+ 24us) client_negotiation.cc:769] Sending connection context
0805 23:14:29.518266 (+ 88us) client_negotiation.cc:240] Negotiation successful
0805 23:14:29.518351 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163}
I20260805 23:14:29.518934 13264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:29.514753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.195:38353 (local address 127.25.92.194:41817)
0805 23:14:29.514887 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0805 23:14:29.514892 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:14:29.515532 (+ 640us) server_negotiation.cc:407] Connection header received
0805 23:14:29.515624 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:14:29.515626 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:14:29.515680 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:14:29.515765 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:14:29.516223 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:29.516756 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:29.517885 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:29.518288 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:29.518431 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:29.518480 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0805 23:14:29.518728 (+ 248us) server_negotiation.cc:299] Negotiation successful
0805 23:14:29.518814 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260805 23:14:29.519392 13274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:29.514611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.194:41817 (local address 127.25.92.195:38353)
0805 23:14:29.515411 (+ 800us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:29.515424 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0805 23:14:29.515517 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:14:29.515912 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:14:29.515915 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:14:29.515936 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:14:29.516079 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:14:29.516086 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:14:29.516883 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:14:29.516888 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:14:29.517740 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:14:29.517751 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:14:29.518428 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:14:29.518432 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:14:29.518535 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:29.518560 (+ 25us) client_negotiation.cc:769] Sending connection context
0805 23:14:29.519148 (+ 588us) client_negotiation.cc:240] Negotiation successful
0805 23:14:29.519239 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":729,"mutex_wait_us":292,"thread_start_us":158,"threads_started":1}
I20260805 23:14:29.519393 12935 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b75c37e7c3ed440c9523223d376c9979" candidate_uuid: "1b477a4ce180447cac00168c49fd41d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12ae1908f90a4ce3a9c7a2dd5a13a170" is_pre_election: true
I20260805 23:14:29.519816 12935 raft_consensus.cc:2454] T b75c37e7c3ed440c9523223d376c9979 P 12ae1908f90a4ce3a9c7a2dd5a13a170 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b477a4ce180447cac00168c49fd41d7 in term 0.
I20260805 23:14:29.520116 13069 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b75c37e7c3ed440c9523223d376c9979" candidate_uuid: "1b477a4ce180447cac00168c49fd41d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed3ccdab406486cb73b637d14de8fb7" is_pre_election: true
I20260805 23:14:29.520418 13069 raft_consensus.cc:2454] T b75c37e7c3ed440c9523223d376c9979 P 4ed3ccdab406486cb73b637d14de8fb7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b477a4ce180447cac00168c49fd41d7 in term 0.
I20260805 23:14:29.520402 13135 leader_election.cc:302] T b75c37e7c3ed440c9523223d376c9979 P 1b477a4ce180447cac00168c49fd41d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12ae1908f90a4ce3a9c7a2dd5a13a170, 1b477a4ce180447cac00168c49fd41d7; no voters:
W20260805 23:14:29.520597 13116 tablet.cc:2369] T b75c37e7c3ed440c9523223d376c9979 P 4ed3ccdab406486cb73b637d14de8fb7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260805 23:14:29.520745 13272 raft_consensus.cc:2781] T b75c37e7c3ed440c9523223d376c9979 P 1b477a4ce180447cac00168c49fd41d7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260805 23:14:29.520834 13272 raft_consensus.cc:482] T b75c37e7c3ed440c9523223d376c9979 P 1b477a4ce180447cac00168c49fd41d7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260805 23:14:29.520885 13272 raft_consensus.cc:3037] T b75c37e7c3ed440c9523223d376c9979 P 1b477a4ce180447cac00168c49fd41d7 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:14:29.522010 13272 raft_consensus.cc:504] T b75c37e7c3ed440c9523223d376c9979 P 1b477a4ce180447cac00168c49fd41d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b477a4ce180447cac00168c49fd41d7" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 46275 } } peers { permanent_uuid: "12ae1908f90a4ce3a9c7a2dd5a13a170" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 44073 } } peers { permanent_uuid: "4ed3ccdab406486cb73b637d14de8fb7" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 41817 } }
I20260805 23:14:29.522405 13272 leader_election.cc:288] T b75c37e7c3ed440c9523223d376c9979 P 1b477a4ce180447cac00168c49fd41d7 [CANDIDATE]: Term 1 election: Requested vote from peers 12ae1908f90a4ce3a9c7a2dd5a13a170 (127.25.92.193:44073), 4ed3ccdab406486cb73b637d14de8fb7 (127.25.92.194:41817)
I20260805 23:14:29.522835 13069 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b75c37e7c3ed440c9523223d376c9979" candidate_uuid: "1b477a4ce180447cac00168c49fd41d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ed3ccdab406486cb73b637d14de8fb7"
I20260805 23:14:29.522835 12935 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b75c37e7c3ed440c9523223d376c9979" candidate_uuid: "1b477a4ce180447cac00168c49fd41d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12ae1908f90a4ce3a9c7a2dd5a13a170"
I20260805 23:14:29.522980 13069 raft_consensus.cc:3037] T b75c37e7c3ed440c9523223d376c9979 P 4ed3ccdab406486cb73b637d14de8fb7 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:14:29.522980 12935 raft_consensus.cc:3037] T b75c37e7c3ed440c9523223d376c9979 P 12ae1908f90a4ce3a9c7a2dd5a13a170 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:14:29.524370 12935 raft_consensus.cc:2454] T b75c37e7c3ed440c9523223d376c9979 P 12ae1908f90a4ce3a9c7a2dd5a13a170 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b477a4ce180447cac00168c49fd41d7 in term 1.
I20260805 23:14:29.524370 13069 raft_consensus.cc:2454] T b75c37e7c3ed440c9523223d376c9979 P 4ed3ccdab406486cb73b637d14de8fb7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b477a4ce180447cac00168c49fd41d7 in term 1.
I20260805 23:14:29.524762 13135 leader_election.cc:302] T b75c37e7c3ed440c9523223d376c9979 P 1b477a4ce180447cac00168c49fd41d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12ae1908f90a4ce3a9c7a2dd5a13a170, 1b477a4ce180447cac00168c49fd41d7; no voters:
I20260805 23:14:29.524962 13272 raft_consensus.cc:2781] T b75c37e7c3ed440c9523223d376c9979 P 1b477a4ce180447cac00168c49fd41d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:14:29.525249 13272 raft_consensus.cc:686] T b75c37e7c3ed440c9523223d376c9979 P 1b477a4ce180447cac00168c49fd41d7 [term 1 LEADER]: Becoming Leader. State: Replica: 1b477a4ce180447cac00168c49fd41d7, State: Running, Role: LEADER
I20260805 23:14:29.525724 13272 consensus_queue.cc:237] T b75c37e7c3ed440c9523223d376c9979 P 1b477a4ce180447cac00168c49fd41d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b477a4ce180447cac00168c49fd41d7" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 46275 } } peers { permanent_uuid: "12ae1908f90a4ce3a9c7a2dd5a13a170" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 44073 } } peers { permanent_uuid: "4ed3ccdab406486cb73b637d14de8fb7" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 41817 } }
I20260805 23:14:29.529072 12791 catalog_manager.cc:5495] T b75c37e7c3ed440c9523223d376c9979 P 1b477a4ce180447cac00168c49fd41d7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b477a4ce180447cac00168c49fd41d7 (127.25.92.195). New cstate: current_term: 1 leader_uuid: "1b477a4ce180447cac00168c49fd41d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b477a4ce180447cac00168c49fd41d7" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 46275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12ae1908f90a4ce3a9c7a2dd5a13a170" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 44073 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4ed3ccdab406486cb73b637d14de8fb7" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 41817 } health_report { overall_health: UNKNOWN } } }
I20260805 23:14:29.582973 13260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:29.578932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56856 (local address 127.25.92.195:46275)
0805 23:14:29.579098 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0805 23:14:29.579102 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:14:29.579124 (+ 22us) server_negotiation.cc:407] Connection header received
0805 23:14:29.579238 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:14:29.579240 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:14:29.579298 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:14:29.579395 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 23:14:29.579684 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:29.580195 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:29.581180 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:29.581401 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:29.581511 (+ 110us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:29.581945 (+ 434us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 23:14:29.582409 (+ 464us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 23:14:29.582561 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0805 23:14:29.582715 (+ 154us) server_negotiation.cc:299] Negotiation successful
0805 23:14:29.582770 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260805 23:14:29.589637 13069 raft_consensus.cc:1261] T b75c37e7c3ed440c9523223d376c9979 P 4ed3ccdab406486cb73b637d14de8fb7 [term 1 FOLLOWER]: Refusing update from remote peer 1b477a4ce180447cac00168c49fd41d7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:14:29.589699 12935 raft_consensus.cc:1261] T b75c37e7c3ed440c9523223d376c9979 P 12ae1908f90a4ce3a9c7a2dd5a13a170 [term 1 FOLLOWER]: Refusing update from remote peer 1b477a4ce180447cac00168c49fd41d7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:14:29.590456 13275 consensus_queue.cc:1035] T b75c37e7c3ed440c9523223d376c9979 P 1b477a4ce180447cac00168c49fd41d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ed3ccdab406486cb73b637d14de8fb7" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 41817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:14:29.590672 13272 consensus_queue.cc:1035] T b75c37e7c3ed440c9523223d376c9979 P 1b477a4ce180447cac00168c49fd41d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12ae1908f90a4ce3a9c7a2dd5a13a170" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 44073 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:14:29.600278 13280 mvcc.cc:205] Tried to move back new op lower bound from 7315339958631186432 to 7315339958380048384. Current Snapshot: MvccSnapshot[applied={T|T < 7315339958631186432}]
I20260805 23:14:29.600411 13279 mvcc.cc:205] Tried to move back new op lower bound from 7315339958631186432 to 7315339958380048384. Current Snapshot: MvccSnapshot[applied={T|T < 7315339958631186432}]
I20260805 23:14:29.609157 12790 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35004:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:14:29.609383 12790 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35004:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:14:29.612224 12790 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 43adec9805c44a508e87f1f9ad914e30: Sending DeleteTablet for 3 replicas of tablet b75c37e7c3ed440c9523223d376c9979
I20260805 23:14:29.613287 12915 tablet_service.cc:1514] Processing DeleteTablet for tablet b75c37e7c3ed440c9523223d376c9979 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:14:29 UTC) from {username='slave'} at 127.0.0.1:59718
I20260805 23:14:29.613351 13048 tablet_service.cc:1514] Processing DeleteTablet for tablet b75c37e7c3ed440c9523223d376c9979 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:14:29 UTC) from {username='slave'} at 127.0.0.1:48704
I20260805 23:14:29.613946 13289 tablet_replica.cc:330] T b75c37e7c3ed440c9523223d376c9979 P 4ed3ccdab406486cb73b637d14de8fb7: stopping tablet replica
I20260805 23:14:29.613912 13181 tablet_service.cc:1514] Processing DeleteTablet for tablet b75c37e7c3ed440c9523223d376c9979 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:14:29 UTC) from {username='slave'} at 127.0.0.1:56852
I20260805 23:14:29.614336 13289 raft_consensus.cc:2229] T b75c37e7c3ed440c9523223d376c9979 P 4ed3ccdab406486cb73b637d14de8fb7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 23:14:29.614431 13290 tablet_replica.cc:330] T b75c37e7c3ed440c9523223d376c9979 P 1b477a4ce180447cac00168c49fd41d7: stopping tablet replica
I20260805 23:14:29.614732 13289 raft_consensus.cc:2258] T b75c37e7c3ed440c9523223d376c9979 P 4ed3ccdab406486cb73b637d14de8fb7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:14:29.614723 13290 raft_consensus.cc:2229] T b75c37e7c3ed440c9523223d376c9979 P 1b477a4ce180447cac00168c49fd41d7 [term 1 LEADER]: Raft consensus shutting down.
I20260805 23:14:29.615100 13290 raft_consensus.cc:2258] T b75c37e7c3ed440c9523223d376c9979 P 1b477a4ce180447cac00168c49fd41d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:14:29.615265 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 12852
I20260805 23:14:29.616676 13289 ts_tablet_manager.cc:1905] T b75c37e7c3ed440c9523223d376c9979 P 4ed3ccdab406486cb73b637d14de8fb7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:14:29.617542 13290 ts_tablet_manager.cc:1905] T b75c37e7c3ed440c9523223d376c9979 P 1b477a4ce180447cac00168c49fd41d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:14:29.619865 13289 ts_tablet_manager.cc:1918] T b75c37e7c3ed440c9523223d376c9979 P 4ed3ccdab406486cb73b637d14de8fb7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 23:14:29.619974 13289 log.cc:1196] T b75c37e7c3ed440c9523223d376c9979 P 4ed3ccdab406486cb73b637d14de8fb7: Deleting WAL directory at /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971406251336-25971-0/minicluster-data/ts-1/wal/wals/b75c37e7c3ed440c9523223d376c9979
I20260805 23:14:29.620349 13289 ts_tablet_manager.cc:1939] T b75c37e7c3ed440c9523223d376c9979 P 4ed3ccdab406486cb73b637d14de8fb7: Deleting consensus metadata
I20260805 23:14:29.620683 13290 ts_tablet_manager.cc:1918] T b75c37e7c3ed440c9523223d376c9979 P 1b477a4ce180447cac00168c49fd41d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 23:14:29.620790 13290 log.cc:1196] T b75c37e7c3ed440c9523223d376c9979 P 1b477a4ce180447cac00168c49fd41d7: Deleting WAL directory at /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971406251336-25971-0/minicluster-data/ts-2/wal/wals/b75c37e7c3ed440c9523223d376c9979
I20260805 23:14:29.621143 13290 ts_tablet_manager.cc:1939] T b75c37e7c3ed440c9523223d376c9979 P 1b477a4ce180447cac00168c49fd41d7: Deleting consensus metadata
I20260805 23:14:29.621955 12777 catalog_manager.cc:4841] TS 4ed3ccdab406486cb73b637d14de8fb7 (127.25.92.194:41817): tablet b75c37e7c3ed440c9523223d376c9979 (table test-table [id=6facdbfdf5094b85854dbf7e49b762af]) successfully deleted
I20260805 23:14:29.622373 12778 catalog_manager.cc:4841] TS 1b477a4ce180447cac00168c49fd41d7 (127.25.92.195:46275): tablet b75c37e7c3ed440c9523223d376c9979 (table test-table [id=6facdbfdf5094b85854dbf7e49b762af]) successfully deleted
I20260805 23:14:29.627616 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 12984
W20260805 23:14:29.627609 12776 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.25.92.193:44073 (error 108)
I20260805 23:14:29.631326 13259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:29.630897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.193:44073 (local address 127.0.0.1:59734)
0805 23:14:29.631098 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:29.631199 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.193:44073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
W20260805 23:14:29.631652 12776 catalog_manager.cc:4584] TS 12ae1908f90a4ce3a9c7a2dd5a13a170 (127.25.92.193:44073): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b75c37e7c3ed440c9523223d376c9979: Network error: Client connection negotiation failed: client connection to 127.25.92.193:44073: connect: Connection refused (error 111)
I20260805 23:14:29.635435 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 13118
I20260805 23:14:29.642100 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 12759
2026-08-05T23:14:29Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1317 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971406251336-25971-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 23:14:29 dist-test-slave-ss4n krb5kdc[13294](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 23:14:29 dist-test-slave-ss4n krb5kdc[13294](info): set up 2 sockets
Aug 05 23:14:29 dist-test-slave-ss4n krb5kdc[13294](info): commencing operation
krb5kdc: starting...
W20260805 23:14:30.727973 25971 mini_kdc.cc:121] Time spent starting KDC: real 1.063s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971406251336-25971-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971406251336-25971-0/krb5kdc/test-user.keytab.
Aug 05 23:14:30 dist-test-slave-ss4n krb5kdc[13294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971670, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T23:14:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:14:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.92.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.92.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.92.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab.
Entry for principal kudu/127.25.92.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab.
I20260805 23:14:30.870306 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971406251336-25971-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971406251336-25971-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971406251336-25971-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.254:45211
--webserver_interface=127.25.92.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab
--principal=kudu/127.25.92.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:37715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.92.254:45211
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:14:30.991814 13308 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:14:30.992208 13308 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:14:30.992296 13308 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:14:30.997267 13308 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:14:30.997380 13308 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:14:30.997418 13308 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:14:30.997511 13308 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:14:30.997576 13308 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:14:31.003499 13308 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:37715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971406251336-25971-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971406251336-25971-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.92.254:45211
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab
--principal=kudu/127.25.92.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.254:45211
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.92.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13308
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971406251336-25971-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:14:31.004719 13308 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:14:31.005877 13308 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:14:31.015479 13317 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:14:31.015483 13314 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:14:31.015538 13308 server_base.cc:1034] running on GCE node
W20260805 23:14:31.015483 13313 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:14:31.016451 13308 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:14:31.017925 13308 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:14:31.019131 13308 hybrid_clock.cc:648] HybridClock initialized: now 1785971671019095 us; error 56 us; skew 500 ppm
Aug 05 23:14:31 dist-test-slave-ss4n krb5kdc[13294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971671, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:14:31.022459 13308 init.cc:377] Logged in from keytab as kudu/127.25.92.254@KRBTEST.COM (short username kudu)
I20260805 23:14:31.023540 13308 webserver.cc:460] Webserver started at http://127.25.92.254:40013/ using document root <none> and password file <none>
I20260805 23:14:31.024237 13308 fs_manager.cc:359] Metadata directory not provided
I20260805 23:14:31.024339 13308 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:14:31.024593 13308 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:14:31.026998 13308 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971406251336-25971-0/minicluster-data/master-0/data/instance:
uuid: "3ba684461f1f4ad0a79c797f551b3aee"
format_stamp: "Formatted at 2026-08-05 23:14:31 on dist-test-slave-ss4n"
server_key: "58ab29aaedae06ed89d1977681c95ecd"
server_key_iv: "f2b177174150db86e92c110da672947a"
server_key_version: "encryptionkey@0"
I20260805 23:14:31.027684 13308 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971406251336-25971-0/minicluster-data/master-0/wal/instance:
uuid: "3ba684461f1f4ad0a79c797f551b3aee"
format_stamp: "Formatted at 2026-08-05 23:14:31 on dist-test-slave-ss4n"
server_key: "58ab29aaedae06ed89d1977681c95ecd"
server_key_iv: "f2b177174150db86e92c110da672947a"
server_key_version: "encryptionkey@0"
I20260805 23:14:31.032480 13308 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260805 23:14:31.036130 13324 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:14:31.037801 13308 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:14:31.037950 13308 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971406251336-25971-0/minicluster-data/master-0/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971406251336-25971-0/minicluster-data/master-0/wal
uuid: "3ba684461f1f4ad0a79c797f551b3aee"
format_stamp: "Formatted at 2026-08-05 23:14:31 on dist-test-slave-ss4n"
server_key: "58ab29aaedae06ed89d1977681c95ecd"
server_key_iv: "f2b177174150db86e92c110da672947a"
server_key_version: "encryptionkey@0"
I20260805 23:14:31.038130 13308 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971406251336-25971-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971406251336-25971-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971406251336-25971-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:14:31.062821 13308 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:14:31.066859 13308 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:14:31.067139 13308 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:14:31.075117 13308 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.254:45211
I20260805 23:14:31.075181 13378 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.254:45211 every 8 connection(s)
I20260805 23:14:31.076216 13308 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
I20260805 23:14:31.079198 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 13308
I20260805 23:14:31.079375 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971406251336-25971-0/minicluster-data/master-0/wal/instance
I20260805 23:14:31.079711 25971 external_mini_cluster.cc:1428] Setting key 72810380c7842cc7a3fbbd5cabe374e7
I20260805 23:14:31.079929 13379 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:14:31.088423 13379 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ba684461f1f4ad0a79c797f551b3aee: Bootstrap starting.
Aug 05 23:14:31 dist-test-slave-ss4n krb5kdc[13294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971670, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM
I20260805 23:14:31.091629 13379 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3ba684461f1f4ad0a79c797f551b3aee: Neither blocks nor log segments found. Creating new log.
I20260805 23:14:31.092878 13379 log.cc:824] T 00000000000000000000000000000000 P 3ba684461f1f4ad0a79c797f551b3aee: Log is configured to *not* fsync() on all Append() calls
I20260805 23:14:31.095706 13379 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ba684461f1f4ad0a79c797f551b3aee: No bootstrap required, opened a new log
I20260805 23:14:31.098467 13382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:31.081466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48536 (local address 127.25.92.254:45211)
0805 23:14:31.082085 (+ 619us) server_negotiation.cc:206] Beginning negotiation
0805 23:14:31.082105 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0805 23:14:31.082156 (+ 51us) server_negotiation.cc:407] Connection header received
0805 23:14:31.082895 (+ 739us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:14:31.082913 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:14:31.083299 (+ 386us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:14:31.083630 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:14:31.084695 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:31.086128 (+ 1433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:31.087392 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:31.088007 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:31.088166 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:31.091980 (+ 3814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:14:31.091990 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:14:31.092034 (+ 44us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:14:31.092088 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:14:31.094883 (+ 2795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:14:31.095534 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:14:31.095542 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:14:31.095549 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:14:31.095705 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:14:31.096084 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:14:31.096089 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:14:31.096091 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:14:31.096595 (+ 504us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:14:31.096825 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0805 23:14:31.097219 (+ 394us) server_negotiation.cc:299] Negotiation successful
0805 23:14:31.097635 (+ 416us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":192,"threads_started":1}
I20260805 23:14:31.099663 13379 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3ba684461f1f4ad0a79c797f551b3aee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ba684461f1f4ad0a79c797f551b3aee" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 45211 } }
I20260805 23:14:31.100046 13379 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3ba684461f1f4ad0a79c797f551b3aee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:14:31.100150 13379 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3ba684461f1f4ad0a79c797f551b3aee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ba684461f1f4ad0a79c797f551b3aee, State: Initialized, Role: FOLLOWER
I20260805 23:14:31.100638 13379 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3ba684461f1f4ad0a79c797f551b3aee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ba684461f1f4ad0a79c797f551b3aee" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 45211 } }
I20260805 23:14:31.100809 13379 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3ba684461f1f4ad0a79c797f551b3aee [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:14:31.100899 13379 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3ba684461f1f4ad0a79c797f551b3aee [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:14:31.101011 13379 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3ba684461f1f4ad0a79c797f551b3aee [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:14:31.102244 13379 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3ba684461f1f4ad0a79c797f551b3aee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ba684461f1f4ad0a79c797f551b3aee" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 45211 } }
I20260805 23:14:31.102662 13379 leader_election.cc:302] T 00000000000000000000000000000000 P 3ba684461f1f4ad0a79c797f551b3aee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3ba684461f1f4ad0a79c797f551b3aee; no voters:
I20260805 23:14:31.103070 13379 leader_election.cc:288] T 00000000000000000000000000000000 P 3ba684461f1f4ad0a79c797f551b3aee [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 23:14:31.103183 13384 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3ba684461f1f4ad0a79c797f551b3aee [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:14:31.103508 13384 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3ba684461f1f4ad0a79c797f551b3aee [term 1 LEADER]: Becoming Leader. State: Replica: 3ba684461f1f4ad0a79c797f551b3aee, State: Running, Role: LEADER
I20260805 23:14:31.103900 13384 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3ba684461f1f4ad0a79c797f551b3aee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ba684461f1f4ad0a79c797f551b3aee" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 45211 } }
I20260805 23:14:31.104486 13379 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3ba684461f1f4ad0a79c797f551b3aee [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:14:31.105679 13386 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ba684461f1f4ad0a79c797f551b3aee [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3ba684461f1f4ad0a79c797f551b3aee. Latest consensus state: current_term: 1 leader_uuid: "3ba684461f1f4ad0a79c797f551b3aee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ba684461f1f4ad0a79c797f551b3aee" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 45211 } } }
I20260805 23:14:31.105832 13386 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ba684461f1f4ad0a79c797f551b3aee [sys.catalog]: This master's current role is: LEADER
I20260805 23:14:31.106259 13393 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:14:31.106235 13385 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ba684461f1f4ad0a79c797f551b3aee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3ba684461f1f4ad0a79c797f551b3aee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ba684461f1f4ad0a79c797f551b3aee" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 45211 } } }
I20260805 23:14:31.106405 13385 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ba684461f1f4ad0a79c797f551b3aee [sys.catalog]: This master's current role is: LEADER
I20260805 23:14:31.109882 13393 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:14:31.116458 13393 catalog_manager.cc:1347] Generated new cluster ID: 5f570b57a93842df80b63c47ac53e5a4
I20260805 23:14:31.116647 13393 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:14:31.136983 13393 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:14:31.138072 13393 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:14:31.150101 13393 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3ba684461f1f4ad0a79c797f551b3aee: Generated new TSK 0
I20260805 23:14:31.150930 13393 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.92.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.92.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.92.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.193.keytab.
Entry for principal kudu/127.25.92.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.193.keytab.
I20260805 23:14:31.185847 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971406251336-25971-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971406251336-25971-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971406251336-25971-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971406251336-25971-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.92.193:0
--local_ip_for_outbound_sockets=127.25.92.193
--webserver_interface=127.25.92.193
--webserver_port=0
--tserver_master_addrs=127.25.92.254:45211
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.193.keytab
--principal=kudu/127.25.92.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:37715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:14:31.333061 13406 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:14:31.333369 13406 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:14:31.333416 13406 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:14:31.337826 13406 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:14:31.337918 13406 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:14:31.337988 13406 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.92.193
I20260805 23:14:31.343930 13406 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:37715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971406251336-25971-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971406251336-25971-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.193.keytab
--principal=kudu/127.25.92.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.92.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971406251336-25971-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.92.193
--webserver_port=0
--tserver_master_addrs=127.25.92.254:45211
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13406
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.92.193
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971406251336-25971-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:14:31.345099 13406 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:14:31.346169 13406 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:14:31.356809 13414 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:14:31.356917 13406 server_base.cc:1034] running on GCE node
W20260805 23:14:31.356762 13411 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:14:31.357146 13412 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:14:31.357664 13406 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:14:31.358453 13406 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:14:31.359669 13406 hybrid_clock.cc:648] HybridClock initialized: now 1785971671359649 us; error 35 us; skew 500 ppm
Aug 05 23:14:31 dist-test-slave-ss4n krb5kdc[13294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971671, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:14:31.363075 13406 init.cc:377] Logged in from keytab as kudu/127.25.92.193@KRBTEST.COM (short username kudu)
I20260805 23:14:31.364185 13406 webserver.cc:460] Webserver started at http://127.25.92.193:33575/ using document root <none> and password file <none>
I20260805 23:14:31.364800 13406 fs_manager.cc:359] Metadata directory not provided
I20260805 23:14:31.364899 13406 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:14:31.365156 13406 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:14:31.367643 13406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971406251336-25971-0/minicluster-data/ts-0/data/instance:
uuid: "85ce884025924a8591d29a026ec74b9e"
format_stamp: "Formatted at 2026-08-05 23:14:31 on dist-test-slave-ss4n"
server_key: "06567700ae15706079f3e00acd8faa49"
server_key_iv: "012a8579f839beca1e43f26126c29b5c"
server_key_version: "encryptionkey@0"
I20260805 23:14:31.368346 13406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971406251336-25971-0/minicluster-data/ts-0/wal/instance:
uuid: "85ce884025924a8591d29a026ec74b9e"
format_stamp: "Formatted at 2026-08-05 23:14:31 on dist-test-slave-ss4n"
server_key: "06567700ae15706079f3e00acd8faa49"
server_key_iv: "012a8579f839beca1e43f26126c29b5c"
server_key_version: "encryptionkey@0"
I20260805 23:14:31.372820 13406 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260805 23:14:31.375948 13421 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:14:31.377671 13406 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:14:31.377882 13406 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971406251336-25971-0/minicluster-data/ts-0/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971406251336-25971-0/minicluster-data/ts-0/wal
uuid: "85ce884025924a8591d29a026ec74b9e"
format_stamp: "Formatted at 2026-08-05 23:14:31 on dist-test-slave-ss4n"
server_key: "06567700ae15706079f3e00acd8faa49"
server_key_iv: "012a8579f839beca1e43f26126c29b5c"
server_key_version: "encryptionkey@0"
I20260805 23:14:31.378063 13406 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971406251336-25971-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971406251336-25971-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971406251336-25971-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:14:31.403112 13406 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:14:31.407383 13406 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:14:31.407651 13406 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:14:31.408353 13406 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:14:31.409308 13406 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:14:31.409395 13406 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:14:31.409474 13406 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:14:31.409567 13406 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260805 23:14:31.420159 13406 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.193:45141
I20260805 23:14:31.420185 13536 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.193:45141 every 8 connection(s)
I20260805 23:14:31.421319 13406 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971406251336-25971-0/minicluster-data/ts-0/data/info.pb
I20260805 23:14:31.424908 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 13406
I20260805 23:14:31.425051 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971406251336-25971-0/minicluster-data/ts-0/wal/instance
I20260805 23:14:31.425325 25971 external_mini_cluster.cc:1428] Setting key 2c7c5d2a843f5a4a53d9ca20e7a58063
Aug 05 23:14:31 dist-test-slave-ss4n krb5kdc[13294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971671, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.193@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.25.92.194@KRBTEST.COM; defaulting to no policy
I20260805 23:14:31.440228 13382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:31.423784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.193:52193 (local address 127.25.92.254:45211)
0805 23:14:31.423978 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0805 23:14:31.423983 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:14:31.424954 (+ 971us) server_negotiation.cc:407] Connection header received
0805 23:14:31.426644 (+ 1690us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:14:31.426647 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:14:31.426703 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:14:31.426789 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:14:31.428780 (+ 1991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:31.429372 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:31.431364 (+ 1992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:31.431712 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:31.431824 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:31.435386 (+ 3562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:14:31.435389 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:14:31.435391 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:14:31.435420 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:14:31.437330 (+ 1910us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:14:31.438133 (+ 803us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:14:31.438139 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:14:31.438142 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:14:31.438253 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:14:31.438761 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:14:31.438765 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:14:31.438766 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:14:31.438991 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:14:31.439092 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0805 23:14:31.439916 (+ 824us) server_negotiation.cc:299] Negotiation successful
0805 23:14:31.440062 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.92.194@KRBTEST.COM" created.
I20260805 23:14:31.441418 13539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:31.424152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:45211 (local address 127.25.92.193:52193)
0805 23:14:31.424770 (+ 618us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:31.424819 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0805 23:14:31.426398 (+ 1579us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:14:31.427001 (+ 603us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:14:31.427036 (+ 35us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:14:31.427741 (+ 705us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:14:31.428546 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:14:31.428570 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:14:31.429582 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:14:31.429589 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:14:31.431169 (+ 1580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:14:31.431185 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:14:31.431917 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:14:31.431924 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:14:31.432178 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:31.432951 (+ 773us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:14:31.432980 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:14:31.435084 (+ 2104us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:14:31.437554 (+ 2470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:14:31.437566 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:14:31.437582 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:14:31.437996 (+ 414us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:14:31.438394 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:14:31.438397 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:14:31.438399 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:14:31.438571 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:14:31.439201 (+ 630us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:14:31.439242 (+ 41us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:14:31.439607 (+ 365us) client_negotiation.cc:769] Sending connection context
0805 23:14:31.439792 (+ 185us) client_negotiation.cc:240] Negotiation successful
0805 23:14:31.440144 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":139,"threads_started":1}
I20260805 23:14:31.443012 13537 heartbeater.cc:347] Connected to a master server at 127.25.92.254:45211
I20260805 23:14:31.443320 13537 heartbeater.cc:464] Registering TS with master...
I20260805 23:14:31.443893 13537 heartbeater.cc:511] Master 127.25.92.254:45211 requested a full tablet report, sending...
I20260805 23:14:31.445778 13340 ts_manager.cc:194] Registered new tserver with Master: 85ce884025924a8591d29a026ec74b9e (127.25.92.193:45141)
I20260805 23:14:31.446976 13340 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.92.193@KRBTEST.COM'} at 127.25.92.193:52193
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.92.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.194.keytab.
Entry for principal kudu/127.25.92.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.194.keytab.
I20260805 23:14:31.461181 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971406251336-25971-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971406251336-25971-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971406251336-25971-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971406251336-25971-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.92.194:0
--local_ip_for_outbound_sockets=127.25.92.194
--webserver_interface=127.25.92.194
--webserver_port=0
--tserver_master_addrs=127.25.92.254:45211
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.194.keytab
--principal=kudu/127.25.92.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:37715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:14:31.580610 13542 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:14:31.580981 13542 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:14:31.581065 13542 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:14:31.585543 13542 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:14:31.585668 13542 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:14:31.585769 13542 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.92.194
I20260805 23:14:31.591576 13542 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:37715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971406251336-25971-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971406251336-25971-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.194.keytab
--principal=kudu/127.25.92.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.92.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971406251336-25971-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.92.194
--webserver_port=0
--tserver_master_addrs=127.25.92.254:45211
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13542
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.92.194
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971406251336-25971-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:14:31.592806 13542 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:14:31.593932 13542 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:14:31.604457 13548 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:14:31.604537 13547 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:14:31.604614 13550 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:14:31.604461 13542 server_base.cc:1034] running on GCE node
I20260805 23:14:31.605265 13542 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:14:31.606187 13542 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:14:31.607391 13542 hybrid_clock.cc:648] HybridClock initialized: now 1785971671607316 us; error 90 us; skew 500 ppm
Aug 05 23:14:31 dist-test-slave-ss4n krb5kdc[13294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971671, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:14:31.610855 13542 init.cc:377] Logged in from keytab as kudu/127.25.92.194@KRBTEST.COM (short username kudu)
I20260805 23:14:31.611932 13542 webserver.cc:460] Webserver started at http://127.25.92.194:44225/ using document root <none> and password file <none>
I20260805 23:14:31.612557 13542 fs_manager.cc:359] Metadata directory not provided
I20260805 23:14:31.612658 13542 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:14:31.612911 13542 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:14:31.615201 13542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971406251336-25971-0/minicluster-data/ts-1/data/instance:
uuid: "2c609aa9e1ba44adae1501b81d613ca6"
format_stamp: "Formatted at 2026-08-05 23:14:31 on dist-test-slave-ss4n"
server_key: "52f1618ff5451e7292649b83f487eaf2"
server_key_iv: "2857359420aaa20c8a64c25daec492a1"
server_key_version: "encryptionkey@0"
I20260805 23:14:31.615804 13542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971406251336-25971-0/minicluster-data/ts-1/wal/instance:
uuid: "2c609aa9e1ba44adae1501b81d613ca6"
format_stamp: "Formatted at 2026-08-05 23:14:31 on dist-test-slave-ss4n"
server_key: "52f1618ff5451e7292649b83f487eaf2"
server_key_iv: "2857359420aaa20c8a64c25daec492a1"
server_key_version: "encryptionkey@0"
I20260805 23:14:31.620108 13542 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260805 23:14:31.623098 13557 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:14:31.624617 13542 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260805 23:14:31.624786 13542 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971406251336-25971-0/minicluster-data/ts-1/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971406251336-25971-0/minicluster-data/ts-1/wal
uuid: "2c609aa9e1ba44adae1501b81d613ca6"
format_stamp: "Formatted at 2026-08-05 23:14:31 on dist-test-slave-ss4n"
server_key: "52f1618ff5451e7292649b83f487eaf2"
server_key_iv: "2857359420aaa20c8a64c25daec492a1"
server_key_version: "encryptionkey@0"
I20260805 23:14:31.624933 13542 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971406251336-25971-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971406251336-25971-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971406251336-25971-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:14:31.643595 13542 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:14:31.647403 13542 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:14:31.647675 13542 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:14:31.648370 13542 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:14:31.649345 13542 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:14:31.649425 13542 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:14:31.649542 13542 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:14:31.649605 13542 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:14:31.659773 13542 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.194:36539
I20260805 23:14:31.659801 13672 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.194:36539 every 8 connection(s)
I20260805 23:14:31.660815 13542 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971406251336-25971-0/minicluster-data/ts-1/data/info.pb
I20260805 23:14:31.669251 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 13542
I20260805 23:14:31.669415 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971406251336-25971-0/minicluster-data/ts-1/wal/instance
I20260805 23:14:31.669775 25971 external_mini_cluster.cc:1428] Setting key 78db4ba5df6f3458b84eb1a9deadc0d8
Aug 05 23:14:31 dist-test-slave-ss4n krb5kdc[13294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971671, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.194@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM
I20260805 23:14:31.678635 13382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:31.663004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.194:52367 (local address 127.25.92.254:45211)
0805 23:14:31.663221 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0805 23:14:31.663225 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:14:31.664276 (+ 1051us) server_negotiation.cc:407] Connection header received
0805 23:14:31.665957 (+ 1681us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:14:31.665961 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:14:31.666032 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:14:31.666147 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:14:31.667571 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:31.668162 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:31.670015 (+ 1853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:31.670375 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:31.670510 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:31.674189 (+ 3679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:14:31.674193 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:14:31.674195 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:14:31.674227 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:14:31.676099 (+ 1872us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:14:31.676822 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:14:31.676825 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:14:31.676827 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:14:31.676914 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:14:31.677329 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:14:31.677333 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:14:31.677334 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:14:31.677626 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:14:31.677744 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0805 23:14:31.678319 (+ 575us) server_negotiation.cc:299] Negotiation successful
0805 23:14:31.678465 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260805 23:14:31.679659 13675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:31.663435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:45211 (local address 127.25.92.194:52367)
0805 23:14:31.664096 (+ 661us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:31.664148 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0805 23:14:31.665706 (+ 1558us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:14:31.666351 (+ 645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:14:31.666359 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:14:31.666822 (+ 463us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:14:31.667385 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:14:31.667401 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:14:31.668327 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:14:31.668337 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:14:31.669819 (+ 1482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:14:31.669833 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:14:31.670556 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:14:31.670559 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:14:31.670804 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:31.671757 (+ 953us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:14:31.671775 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:14:31.673919 (+ 2144us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:14:31.676309 (+ 2390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:14:31.676316 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:14:31.676337 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:14:31.676677 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:14:31.677025 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:14:31.677028 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:14:31.677030 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:14:31.677207 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:14:31.677742 (+ 535us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:14:31.677752 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:14:31.678046 (+ 294us) client_negotiation.cc:769] Sending connection context
0805 23:14:31.678257 (+ 211us) client_negotiation.cc:240] Negotiation successful
0805 23:14:31.678631 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":147,"threads_started":1}
WARNING: no policy specified for kudu/127.25.92.195@KRBTEST.COM; defaulting to no policy
I20260805 23:14:31.681591 13673 heartbeater.cc:347] Connected to a master server at 127.25.92.254:45211
I20260805 23:14:31.681936 13673 heartbeater.cc:464] Registering TS with master...
I20260805 23:14:31.682591 13673 heartbeater.cc:511] Master 127.25.92.254:45211 requested a full tablet report, sending...
I20260805 23:14:31.684062 13340 ts_manager.cc:194] Registered new tserver with Master: 2c609aa9e1ba44adae1501b81d613ca6 (127.25.92.194:36539)
I20260805 23:14:31.684787 13340 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.92.194@KRBTEST.COM'} at 127.25.92.194:52367
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.92.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.92.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.195.keytab.
Entry for principal kudu/127.25.92.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.195.keytab.
I20260805 23:14:31.706188 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971406251336-25971-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971406251336-25971-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971406251336-25971-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971406251336-25971-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.92.195:0
--local_ip_for_outbound_sockets=127.25.92.195
--webserver_interface=127.25.92.195
--webserver_port=0
--tserver_master_addrs=127.25.92.254:45211
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.195.keytab
--principal=kudu/127.25.92.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:37715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:14:31.825241 13678 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:14:31.825613 13678 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:14:31.825666 13678 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:14:31.830628 13678 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:14:31.830725 13678 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:14:31.830799 13678 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.92.195
I20260805 23:14:31.837237 13678 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:37715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971406251336-25971-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971406251336-25971-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.195.keytab
--principal=kudu/127.25.92.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.92.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971406251336-25971-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.92.195
--webserver_port=0
--tserver_master_addrs=127.25.92.254:45211
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13678
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.92.195
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971406251336-25971-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:14:31.838460 13678 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:14:31.839655 13678 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:14:31.850636 13683 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:14:31.850636 13686 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:14:31.850636 13684 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:14:31.851163 13678 server_base.cc:1034] running on GCE node
I20260805 23:14:31.851792 13678 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:14:31.852514 13678 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:14:31.853715 13678 hybrid_clock.cc:648] HybridClock initialized: now 1785971671853684 us; error 50 us; skew 500 ppm
Aug 05 23:14:31 dist-test-slave-ss4n krb5kdc[13294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971671, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:14:31.857187 13678 init.cc:377] Logged in from keytab as kudu/127.25.92.195@KRBTEST.COM (short username kudu)
I20260805 23:14:31.858485 13678 webserver.cc:460] Webserver started at http://127.25.92.195:45717/ using document root <none> and password file <none>
I20260805 23:14:31.859193 13678 fs_manager.cc:359] Metadata directory not provided
I20260805 23:14:31.859278 13678 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:14:31.859532 13678 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:14:31.862090 13678 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971406251336-25971-0/minicluster-data/ts-2/data/instance:
uuid: "d6b88b1a75084198976f02d157d598b8"
format_stamp: "Formatted at 2026-08-05 23:14:31 on dist-test-slave-ss4n"
server_key: "0c3868dee5514e0eaa09d9f133c3901f"
server_key_iv: "7ff51c400207a4615e8e153e7f2432df"
server_key_version: "encryptionkey@0"
I20260805 23:14:31.862787 13678 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971406251336-25971-0/minicluster-data/ts-2/wal/instance:
uuid: "d6b88b1a75084198976f02d157d598b8"
format_stamp: "Formatted at 2026-08-05 23:14:31 on dist-test-slave-ss4n"
server_key: "0c3868dee5514e0eaa09d9f133c3901f"
server_key_iv: "7ff51c400207a4615e8e153e7f2432df"
server_key_version: "encryptionkey@0"
I20260805 23:14:31.867296 13678 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260805 23:14:31.870191 13693 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:14:31.871764 13678 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:14:31.871963 13678 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971406251336-25971-0/minicluster-data/ts-2/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971406251336-25971-0/minicluster-data/ts-2/wal
uuid: "d6b88b1a75084198976f02d157d598b8"
format_stamp: "Formatted at 2026-08-05 23:14:31 on dist-test-slave-ss4n"
server_key: "0c3868dee5514e0eaa09d9f133c3901f"
server_key_iv: "7ff51c400207a4615e8e153e7f2432df"
server_key_version: "encryptionkey@0"
I20260805 23:14:31.872133 13678 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971406251336-25971-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971406251336-25971-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971406251336-25971-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:14:31.887207 13678 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:14:31.890929 13678 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:14:31.891194 13678 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:14:31.891877 13678 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:14:31.892812 13678 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:14:31.892894 13678 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:14:31.892972 13678 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:14:31.893029 13678 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:14:31.903541 13678 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.195:42799
I20260805 23:14:31.903581 13807 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.195:42799 every 8 connection(s)
I20260805 23:14:31.904709 13678 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971406251336-25971-0/minicluster-data/ts-2/data/info.pb
I20260805 23:14:31.914544 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 13678
I20260805 23:14:31.914767 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971406251336-25971-0/minicluster-data/ts-2/wal/instance
I20260805 23:14:31.915092 25971 external_mini_cluster.cc:1428] Setting key 261242f4cf7b64248023f3db19e9ba35
Aug 05 23:14:31 dist-test-slave-ss4n krb5kdc[13294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971671, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.195@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM
I20260805 23:14:31.921842 13382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:31.907081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.195:59773 (local address 127.25.92.254:45211)
0805 23:14:31.907249 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0805 23:14:31.907253 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:14:31.908194 (+ 941us) server_negotiation.cc:407] Connection header received
0805 23:14:31.909643 (+ 1449us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:14:31.909647 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:14:31.909721 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:14:31.909836 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:14:31.911086 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:31.911631 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:31.913040 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:31.913278 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:31.913371 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:31.917069 (+ 3698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:14:31.917075 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:14:31.917082 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:14:31.917118 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:14:31.919191 (+ 2073us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:14:31.919858 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:14:31.919862 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:14:31.919864 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:14:31.919961 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:14:31.920441 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:14:31.920444 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:14:31.920446 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:14:31.920724 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:14:31.920831 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0805 23:14:31.921432 (+ 601us) server_negotiation.cc:299] Negotiation successful
0805 23:14:31.921621 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260805 23:14:31.922901 13810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:31.907516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:45211 (local address 127.25.92.195:59773)
0805 23:14:31.908024 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:31.908071 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0805 23:14:31.909340 (+ 1269us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:14:31.910023 (+ 683us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:14:31.910031 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:14:31.910433 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:14:31.910922 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:14:31.910936 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:14:31.911848 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:14:31.911852 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:14:31.912885 (+ 1033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:14:31.912898 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:14:31.913578 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:14:31.913588 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:14:31.913767 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:31.914521 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:14:31.914545 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:14:31.916829 (+ 2284us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:14:31.919341 (+ 2512us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:14:31.919348 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:14:31.919368 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:14:31.919732 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:14:31.920066 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:14:31.920070 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:14:31.920072 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:14:31.920302 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:14:31.920840 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:14:31.920846 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:14:31.921123 (+ 277us) client_negotiation.cc:769] Sending connection context
0805 23:14:31.921372 (+ 249us) client_negotiation.cc:240] Negotiation successful
0805 23:14:31.921770 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":133,"threads_started":1}
I20260805 23:14:31.924391 13808 heartbeater.cc:347] Connected to a master server at 127.25.92.254:45211
I20260805 23:14:31.924721 13808 heartbeater.cc:464] Registering TS with master...
I20260805 23:14:31.925398 13808 heartbeater.cc:511] Master 127.25.92.254:45211 requested a full tablet report, sending...
I20260805 23:14:31.926786 13340 ts_manager.cc:194] Registered new tserver with Master: d6b88b1a75084198976f02d157d598b8 (127.25.92.195:42799)
I20260805 23:14:31.927438 13340 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.92.195@KRBTEST.COM'} at 127.25.92.195:59773
I20260805 23:14:31.929641 25971 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260805 23:14:31.940713 13382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:31.931865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35478 (local address 127.25.92.254:45211)
0805 23:14:31.931995 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0805 23:14:31.931998 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 23:14:31.932122 (+ 124us) server_negotiation.cc:407] Connection header received
0805 23:14:31.932227 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:14:31.932229 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:14:31.932274 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:14:31.932352 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:14:31.933102 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:31.933703 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:31.934659 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:31.934876 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:31.934985 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:31.936795 (+ 1810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:14:31.936799 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:14:31.936809 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:14:31.936837 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:14:31.938721 (+ 1884us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:14:31.939253 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:14:31.939256 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:14:31.939257 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:14:31.939343 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:14:31.939703 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:14:31.939706 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:14:31.939707 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:14:31.939970 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:14:31.940085 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0805 23:14:31.940427 (+ 342us) server_negotiation.cc:299] Negotiation successful
0805 23:14:31.940548 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260805 23:14:31.944293 13340 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35478:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260805 23:14:31.947619 13340 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260805 23:14:31.965121 13822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:31.957693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.195:42799 (local address 127.0.0.1:43040)
0805 23:14:31.958751 (+ 1058us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:31.958816 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0805 23:14:31.958966 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:14:31.959697 (+ 731us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:14:31.959703 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:14:31.959727 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:14:31.959887 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:14:31.959921 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:14:31.961298 (+ 1377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:14:31.961302 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:14:31.963257 (+ 1955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:14:31.963264 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:14:31.964374 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:14:31.964378 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:14:31.964508 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:31.964562 (+ 54us) client_negotiation.cc:769] Sending connection context
0805 23:14:31.964725 (+ 163us) client_negotiation.cc:240] Negotiation successful
0805 23:14:31.964942 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":848,"mutex_wait_us":249,"thread_start_us":120,"threads_started":1}
I20260805 23:14:31.965531 13824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:31.958726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47198 (local address 127.25.92.193:45141)
0805 23:14:31.959145 (+ 419us) server_negotiation.cc:206] Beginning negotiation
0805 23:14:31.959153 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0805 23:14:31.959406 (+ 253us) server_negotiation.cc:407] Connection header received
0805 23:14:31.959598 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:14:31.959604 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:14:31.959811 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:14:31.959969 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:14:31.960269 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:31.961296 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:31.963419 (+ 2123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:31.964455 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:31.964566 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:31.964720 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0805 23:14:31.965206 (+ 486us) server_negotiation.cc:299] Negotiation successful
0805 23:14:31.965337 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":310,"thread_start_us":108,"threads_started":1}
I20260805 23:14:31.965849 13819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:31.958102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.194:36539 (local address 127.0.0.1:42820)
0805 23:14:31.959670 (+ 1568us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:31.959684 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0805 23:14:31.959764 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:14:31.960492 (+ 728us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:14:31.960495 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:14:31.960526 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:14:31.960649 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:14:31.960656 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:14:31.962156 (+ 1500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:14:31.962160 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:14:31.963290 (+ 1130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:14:31.963303 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:14:31.964797 (+ 1494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:14:31.964801 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:14:31.964980 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:31.965030 (+ 50us) client_negotiation.cc:769] Sending connection context
0805 23:14:31.965113 (+ 83us) client_negotiation.cc:240] Negotiation successful
0805 23:14:31.965696 (+ 583us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1506,"mutex_wait_us":48,"thread_start_us":101,"threads_started":1}
I20260805 23:14:31.965981 13821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:31.957824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43040 (local address 127.25.92.195:42799)
0805 23:14:31.959173 (+ 1349us) server_negotiation.cc:206] Beginning negotiation
0805 23:14:31.959183 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0805 23:14:31.959205 (+ 22us) server_negotiation.cc:407] Connection header received
0805 23:14:31.959272 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:14:31.959276 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:14:31.959521 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:14:31.959707 (+ 186us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:14:31.960068 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:31.961136 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:31.963421 (+ 2285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:31.964220 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:31.964373 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:31.964557 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0805 23:14:31.964761 (+ 204us) server_negotiation.cc:299] Negotiation successful
0805 23:14:31.964886 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1226,"thread_start_us":201,"threads_started":1}
I20260805 23:14:31.966302 13823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:31.958452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42820 (local address 127.25.92.194:36539)
0805 23:14:31.958915 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0805 23:14:31.958928 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0805 23:14:31.959871 (+ 943us) server_negotiation.cc:407] Connection header received
0805 23:14:31.959959 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:14:31.959964 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:14:31.960140 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:14:31.960320 (+ 180us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:14:31.960798 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:31.961978 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:31.963483 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:31.964421 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:31.964566 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:31.964721 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0805 23:14:31.965384 (+ 663us) server_negotiation.cc:299] Negotiation successful
0805 23:14:31.965556 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":123,"threads_started":1}
I20260805 23:14:31.965893 13820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:31.957725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.193:45141 (local address 127.0.0.1:47198)
0805 23:14:31.959295 (+ 1570us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:31.959311 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0805 23:14:31.959430 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:14:31.959963 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:14:31.959969 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:14:31.960005 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:14:31.960133 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:14:31.960140 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:14:31.961439 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:14:31.961442 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:14:31.963236 (+ 1794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:14:31.963251 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:14:31.964797 (+ 1546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:14:31.964801 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:14:31.965009 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:31.965041 (+ 32us) client_negotiation.cc:769] Sending connection context
0805 23:14:31.965113 (+ 72us) client_negotiation.cc:240] Negotiation successful
0805 23:14:31.965776 (+ 663us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1503,"mutex_wait_us":694,"thread_start_us":95,"threads_started":1}
I20260805 23:14:31.969010 13741 tablet_service.cc:1467] Processing CreateTablet for tablet 70995a7bad84452ebf73adabad1e8e26 (DEFAULT_TABLE table=test-table [id=45c111484ade465f88a68c38d83dbce5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:14:31.969450 13605 tablet_service.cc:1467] Processing CreateTablet for tablet 70995a7bad84452ebf73adabad1e8e26 (DEFAULT_TABLE table=test-table [id=45c111484ade465f88a68c38d83dbce5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:14:31.970335 13741 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 70995a7bad84452ebf73adabad1e8e26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:14:31.970034 13469 tablet_service.cc:1467] Processing CreateTablet for tablet 70995a7bad84452ebf73adabad1e8e26 (DEFAULT_TABLE table=test-table [id=45c111484ade465f88a68c38d83dbce5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:14:31.970510 13605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 70995a7bad84452ebf73adabad1e8e26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:14:31.970989 13469 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 70995a7bad84452ebf73adabad1e8e26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:14:31.977249 13825 tablet_bootstrap.cc:492] T 70995a7bad84452ebf73adabad1e8e26 P d6b88b1a75084198976f02d157d598b8: Bootstrap starting.
I20260805 23:14:31.978711 13826 tablet_bootstrap.cc:492] T 70995a7bad84452ebf73adabad1e8e26 P 85ce884025924a8591d29a026ec74b9e: Bootstrap starting.
I20260805 23:14:31.978711 13827 tablet_bootstrap.cc:492] T 70995a7bad84452ebf73adabad1e8e26 P 2c609aa9e1ba44adae1501b81d613ca6: Bootstrap starting.
I20260805 23:14:31.979739 13825 tablet_bootstrap.cc:654] T 70995a7bad84452ebf73adabad1e8e26 P d6b88b1a75084198976f02d157d598b8: Neither blocks nor log segments found. Creating new log.
I20260805 23:14:31.980975 13827 tablet_bootstrap.cc:654] T 70995a7bad84452ebf73adabad1e8e26 P 2c609aa9e1ba44adae1501b81d613ca6: Neither blocks nor log segments found. Creating new log.
I20260805 23:14:31.980975 13826 tablet_bootstrap.cc:654] T 70995a7bad84452ebf73adabad1e8e26 P 85ce884025924a8591d29a026ec74b9e: Neither blocks nor log segments found. Creating new log.
I20260805 23:14:31.981074 13825 log.cc:824] T 70995a7bad84452ebf73adabad1e8e26 P d6b88b1a75084198976f02d157d598b8: Log is configured to *not* fsync() on all Append() calls
I20260805 23:14:31.981900 13826 log.cc:824] T 70995a7bad84452ebf73adabad1e8e26 P 85ce884025924a8591d29a026ec74b9e: Log is configured to *not* fsync() on all Append() calls
I20260805 23:14:31.982282 13827 log.cc:824] T 70995a7bad84452ebf73adabad1e8e26 P 2c609aa9e1ba44adae1501b81d613ca6: Log is configured to *not* fsync() on all Append() calls
I20260805 23:14:31.983426 13825 tablet_bootstrap.cc:492] T 70995a7bad84452ebf73adabad1e8e26 P d6b88b1a75084198976f02d157d598b8: No bootstrap required, opened a new log
I20260805 23:14:31.983645 13825 ts_tablet_manager.cc:1397] T 70995a7bad84452ebf73adabad1e8e26 P d6b88b1a75084198976f02d157d598b8: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260805 23:14:31.983850 13826 tablet_bootstrap.cc:492] T 70995a7bad84452ebf73adabad1e8e26 P 85ce884025924a8591d29a026ec74b9e: No bootstrap required, opened a new log
I20260805 23:14:31.983995 13826 ts_tablet_manager.cc:1397] T 70995a7bad84452ebf73adabad1e8e26 P 85ce884025924a8591d29a026ec74b9e: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260805 23:14:31.986941 13827 tablet_bootstrap.cc:492] T 70995a7bad84452ebf73adabad1e8e26 P 2c609aa9e1ba44adae1501b81d613ca6: No bootstrap required, opened a new log
I20260805 23:14:31.987206 13827 ts_tablet_manager.cc:1397] T 70995a7bad84452ebf73adabad1e8e26 P 2c609aa9e1ba44adae1501b81d613ca6: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260805 23:14:31.987210 13825 raft_consensus.cc:348] T 70995a7bad84452ebf73adabad1e8e26 P d6b88b1a75084198976f02d157d598b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6b88b1a75084198976f02d157d598b8" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 42799 } } peers { permanent_uuid: "85ce884025924a8591d29a026ec74b9e" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 45141 } } peers { permanent_uuid: "2c609aa9e1ba44adae1501b81d613ca6" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 36539 } }
I20260805 23:14:31.987516 13825 raft_consensus.cc:374] T 70995a7bad84452ebf73adabad1e8e26 P d6b88b1a75084198976f02d157d598b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:14:31.987321 13826 raft_consensus.cc:348] T 70995a7bad84452ebf73adabad1e8e26 P 85ce884025924a8591d29a026ec74b9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6b88b1a75084198976f02d157d598b8" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 42799 } } peers { permanent_uuid: "85ce884025924a8591d29a026ec74b9e" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 45141 } } peers { permanent_uuid: "2c609aa9e1ba44adae1501b81d613ca6" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 36539 } }
I20260805 23:14:31.987577 13825 raft_consensus.cc:729] T 70995a7bad84452ebf73adabad1e8e26 P d6b88b1a75084198976f02d157d598b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6b88b1a75084198976f02d157d598b8, State: Initialized, Role: FOLLOWER
I20260805 23:14:31.987624 13826 raft_consensus.cc:374] T 70995a7bad84452ebf73adabad1e8e26 P 85ce884025924a8591d29a026ec74b9e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:14:31.987686 13826 raft_consensus.cc:729] T 70995a7bad84452ebf73adabad1e8e26 P 85ce884025924a8591d29a026ec74b9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85ce884025924a8591d29a026ec74b9e, State: Initialized, Role: FOLLOWER
I20260805 23:14:31.988090 13825 consensus_queue.cc:260] T 70995a7bad84452ebf73adabad1e8e26 P d6b88b1a75084198976f02d157d598b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6b88b1a75084198976f02d157d598b8" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 42799 } } peers { permanent_uuid: "85ce884025924a8591d29a026ec74b9e" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 45141 } } peers { permanent_uuid: "2c609aa9e1ba44adae1501b81d613ca6" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 36539 } }
I20260805 23:14:31.988095 13826 consensus_queue.cc:260] T 70995a7bad84452ebf73adabad1e8e26 P 85ce884025924a8591d29a026ec74b9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6b88b1a75084198976f02d157d598b8" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 42799 } } peers { permanent_uuid: "85ce884025924a8591d29a026ec74b9e" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 45141 } } peers { permanent_uuid: "2c609aa9e1ba44adae1501b81d613ca6" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 36539 } }
I20260805 23:14:31.989146 13825 ts_tablet_manager.cc:1428] T 70995a7bad84452ebf73adabad1e8e26 P d6b88b1a75084198976f02d157d598b8: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260805 23:14:31.989144 13826 ts_tablet_manager.cc:1428] T 70995a7bad84452ebf73adabad1e8e26 P 85ce884025924a8591d29a026ec74b9e: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260805 23:14:31.989308 13537 heartbeater.cc:503] Master 127.25.92.254:45211 was elected leader, sending a full tablet report...
I20260805 23:14:31.989507 13808 heartbeater.cc:503] Master 127.25.92.254:45211 was elected leader, sending a full tablet report...
I20260805 23:14:31.991258 13827 raft_consensus.cc:348] T 70995a7bad84452ebf73adabad1e8e26 P 2c609aa9e1ba44adae1501b81d613ca6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6b88b1a75084198976f02d157d598b8" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 42799 } } peers { permanent_uuid: "85ce884025924a8591d29a026ec74b9e" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 45141 } } peers { permanent_uuid: "2c609aa9e1ba44adae1501b81d613ca6" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 36539 } }
I20260805 23:14:31.991519 13827 raft_consensus.cc:374] T 70995a7bad84452ebf73adabad1e8e26 P 2c609aa9e1ba44adae1501b81d613ca6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:14:31.991585 13827 raft_consensus.cc:729] T 70995a7bad84452ebf73adabad1e8e26 P 2c609aa9e1ba44adae1501b81d613ca6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c609aa9e1ba44adae1501b81d613ca6, State: Initialized, Role: FOLLOWER
I20260805 23:14:31.992063 13827 consensus_queue.cc:260] T 70995a7bad84452ebf73adabad1e8e26 P 2c609aa9e1ba44adae1501b81d613ca6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6b88b1a75084198976f02d157d598b8" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 42799 } } peers { permanent_uuid: "85ce884025924a8591d29a026ec74b9e" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 45141 } } peers { permanent_uuid: "2c609aa9e1ba44adae1501b81d613ca6" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 36539 } }
I20260805 23:14:31.992805 13673 heartbeater.cc:503] Master 127.25.92.254:45211 was elected leader, sending a full tablet report...
I20260805 23:14:31.993106 13827 ts_tablet_manager.cc:1428] T 70995a7bad84452ebf73adabad1e8e26 P 2c609aa9e1ba44adae1501b81d613ca6: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
W20260805 23:14:32.156028 13809 tablet.cc:2369] T 70995a7bad84452ebf73adabad1e8e26 P d6b88b1a75084198976f02d157d598b8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260805 23:14:32.162111 13674 tablet.cc:2369] T 70995a7bad84452ebf73adabad1e8e26 P 2c609aa9e1ba44adae1501b81d613ca6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260805 23:14:32.172967 13538 tablet.cc:2369] T 70995a7bad84452ebf73adabad1e8e26 P 85ce884025924a8591d29a026ec74b9e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260805 23:14:32.213611 13832 raft_consensus.cc:482] T 70995a7bad84452ebf73adabad1e8e26 P d6b88b1a75084198976f02d157d598b8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 23:14:32.213817 13832 raft_consensus.cc:504] T 70995a7bad84452ebf73adabad1e8e26 P d6b88b1a75084198976f02d157d598b8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6b88b1a75084198976f02d157d598b8" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 42799 } } peers { permanent_uuid: "85ce884025924a8591d29a026ec74b9e" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 45141 } } peers { permanent_uuid: "2c609aa9e1ba44adae1501b81d613ca6" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 36539 } }
I20260805 23:14:32.215023 13832 leader_election.cc:288] T 70995a7bad84452ebf73adabad1e8e26 P d6b88b1a75084198976f02d157d598b8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 85ce884025924a8591d29a026ec74b9e (127.25.92.193:45141), 2c609aa9e1ba44adae1501b81d613ca6 (127.25.92.194:36539)
I20260805 23:14:32.218894 13823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:32.215420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.195:33737 (local address 127.25.92.194:36539)
0805 23:14:32.215568 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0805 23:14:32.215575 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0805 23:14:32.215594 (+ 19us) server_negotiation.cc:407] Connection header received
0805 23:14:32.215655 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:14:32.215663 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:14:32.215705 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:14:32.215803 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:14:32.216181 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:32.216743 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:32.217907 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:32.218307 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:32.218446 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:32.218506 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0805 23:14:32.218683 (+ 177us) server_negotiation.cc:299] Negotiation successful
0805 23:14:32.218761 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260805 23:14:32.218894 13810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:32.215289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.194:36539 (local address 127.25.92.195:33737)
0805 23:14:32.215421 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:32.215440 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0805 23:14:32.215546 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:14:32.215815 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:14:32.215817 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:14:32.215833 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:14:32.215957 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:14:32.215963 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:14:32.216870 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:14:32.216875 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:14:32.217776 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:14:32.217786 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:14:32.218441 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:14:32.218445 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:14:32.218559 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:32.218578 (+ 19us) client_negotiation.cc:769] Sending connection context
0805 23:14:32.218666 (+ 88us) client_negotiation.cc:240] Negotiation successful
0805 23:14:32.218748 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260805 23:14:32.219230 13834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:32.215290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.193:45141 (local address 127.25.92.195:49071)
0805 23:14:32.215931 (+ 641us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:32.215943 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0805 23:14:32.216073 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:14:32.216356 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:14:32.216359 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:14:32.216375 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:14:32.216506 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:14:32.216516 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:14:32.217250 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:14:32.217254 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:14:32.218103 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:14:32.218112 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:14:32.218773 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:14:32.218776 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:14:32.218873 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:32.218900 (+ 27us) client_negotiation.cc:769] Sending connection context
0805 23:14:32.218987 (+ 87us) client_negotiation.cc:240] Negotiation successful
0805 23:14:32.219080 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":590,"mutex_wait_us":192,"thread_start_us":195,"threads_started":1}
I20260805 23:14:32.219733 13824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:32.215454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.195:49071 (local address 127.25.92.193:45141)
0805 23:14:32.215580 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0805 23:14:32.215584 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:14:32.216045 (+ 461us) server_negotiation.cc:407] Connection header received
0805 23:14:32.216200 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:14:32.216205 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:14:32.216256 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:14:32.216346 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:14:32.216631 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:32.217125 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:32.218233 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:32.218649 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:32.218784 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:32.218834 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0805 23:14:32.219512 (+ 678us) server_negotiation.cc:299] Negotiation successful
0805 23:14:32.219604 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260805 23:14:32.219733 13627 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70995a7bad84452ebf73adabad1e8e26" candidate_uuid: "d6b88b1a75084198976f02d157d598b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c609aa9e1ba44adae1501b81d613ca6" is_pre_election: true
I20260805 23:14:32.220077 13627 raft_consensus.cc:2454] T 70995a7bad84452ebf73adabad1e8e26 P 2c609aa9e1ba44adae1501b81d613ca6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6b88b1a75084198976f02d157d598b8 in term 0.
I20260805 23:14:32.220377 13491 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70995a7bad84452ebf73adabad1e8e26" candidate_uuid: "d6b88b1a75084198976f02d157d598b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85ce884025924a8591d29a026ec74b9e" is_pre_election: true
I20260805 23:14:32.220678 13696 leader_election.cc:302] T 70995a7bad84452ebf73adabad1e8e26 P d6b88b1a75084198976f02d157d598b8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c609aa9e1ba44adae1501b81d613ca6, d6b88b1a75084198976f02d157d598b8; no voters:
I20260805 23:14:32.220691 13491 raft_consensus.cc:2454] T 70995a7bad84452ebf73adabad1e8e26 P 85ce884025924a8591d29a026ec74b9e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6b88b1a75084198976f02d157d598b8 in term 0.
I20260805 23:14:32.220978 13832 raft_consensus.cc:2781] T 70995a7bad84452ebf73adabad1e8e26 P d6b88b1a75084198976f02d157d598b8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260805 23:14:32.221077 13832 raft_consensus.cc:482] T 70995a7bad84452ebf73adabad1e8e26 P d6b88b1a75084198976f02d157d598b8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260805 23:14:32.221135 13832 raft_consensus.cc:3037] T 70995a7bad84452ebf73adabad1e8e26 P d6b88b1a75084198976f02d157d598b8 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:14:32.222272 13832 raft_consensus.cc:504] T 70995a7bad84452ebf73adabad1e8e26 P d6b88b1a75084198976f02d157d598b8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6b88b1a75084198976f02d157d598b8" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 42799 } } peers { permanent_uuid: "85ce884025924a8591d29a026ec74b9e" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 45141 } } peers { permanent_uuid: "2c609aa9e1ba44adae1501b81d613ca6" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 36539 } }
I20260805 23:14:32.222667 13832 leader_election.cc:288] T 70995a7bad84452ebf73adabad1e8e26 P d6b88b1a75084198976f02d157d598b8 [CANDIDATE]: Term 1 election: Requested vote from peers 85ce884025924a8591d29a026ec74b9e (127.25.92.193:45141), 2c609aa9e1ba44adae1501b81d613ca6 (127.25.92.194:36539)
I20260805 23:14:32.223054 13491 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70995a7bad84452ebf73adabad1e8e26" candidate_uuid: "d6b88b1a75084198976f02d157d598b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85ce884025924a8591d29a026ec74b9e"
I20260805 23:14:32.223253 13627 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70995a7bad84452ebf73adabad1e8e26" candidate_uuid: "d6b88b1a75084198976f02d157d598b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c609aa9e1ba44adae1501b81d613ca6"
I20260805 23:14:32.223248 13491 raft_consensus.cc:3037] T 70995a7bad84452ebf73adabad1e8e26 P 85ce884025924a8591d29a026ec74b9e [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:14:32.223407 13627 raft_consensus.cc:3037] T 70995a7bad84452ebf73adabad1e8e26 P 2c609aa9e1ba44adae1501b81d613ca6 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:14:32.224900 13491 raft_consensus.cc:2454] T 70995a7bad84452ebf73adabad1e8e26 P 85ce884025924a8591d29a026ec74b9e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6b88b1a75084198976f02d157d598b8 in term 1.
I20260805 23:14:32.224900 13627 raft_consensus.cc:2454] T 70995a7bad84452ebf73adabad1e8e26 P 2c609aa9e1ba44adae1501b81d613ca6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6b88b1a75084198976f02d157d598b8 in term 1.
I20260805 23:14:32.225430 13695 leader_election.cc:302] T 70995a7bad84452ebf73adabad1e8e26 P d6b88b1a75084198976f02d157d598b8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85ce884025924a8591d29a026ec74b9e, d6b88b1a75084198976f02d157d598b8; no voters:
I20260805 23:14:32.225754 13832 raft_consensus.cc:2781] T 70995a7bad84452ebf73adabad1e8e26 P d6b88b1a75084198976f02d157d598b8 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:14:32.226024 13832 raft_consensus.cc:686] T 70995a7bad84452ebf73adabad1e8e26 P d6b88b1a75084198976f02d157d598b8 [term 1 LEADER]: Becoming Leader. State: Replica: d6b88b1a75084198976f02d157d598b8, State: Running, Role: LEADER
I20260805 23:14:32.226487 13832 consensus_queue.cc:237] T 70995a7bad84452ebf73adabad1e8e26 P d6b88b1a75084198976f02d157d598b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6b88b1a75084198976f02d157d598b8" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 42799 } } peers { permanent_uuid: "85ce884025924a8591d29a026ec74b9e" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 45141 } } peers { permanent_uuid: "2c609aa9e1ba44adae1501b81d613ca6" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 36539 } }
I20260805 23:14:32.230571 13341 catalog_manager.cc:5495] T 70995a7bad84452ebf73adabad1e8e26 P d6b88b1a75084198976f02d157d598b8 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6b88b1a75084198976f02d157d598b8 (127.25.92.195). New cstate: current_term: 1 leader_uuid: "d6b88b1a75084198976f02d157d598b8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6b88b1a75084198976f02d157d598b8" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 42799 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85ce884025924a8591d29a026ec74b9e" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 45141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c609aa9e1ba44adae1501b81d613ca6" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 36539 } health_report { overall_health: UNKNOWN } } }
I20260805 23:14:32.263657 13821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:32.259085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43048 (local address 127.25.92.195:42799)
0805 23:14:32.259266 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0805 23:14:32.259269 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 23:14:32.259286 (+ 17us) server_negotiation.cc:407] Connection header received
0805 23:14:32.259372 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:14:32.259374 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:14:32.259432 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:14:32.259535 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 23:14:32.259827 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:32.260330 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:32.261359 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:32.261710 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:32.261806 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:32.262173 (+ 367us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 23:14:32.262929 (+ 756us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 23:14:32.263049 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0805 23:14:32.263234 (+ 185us) server_negotiation.cc:299] Negotiation successful
0805 23:14:32.263300 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260805 23:14:32.270372 13491 raft_consensus.cc:1261] T 70995a7bad84452ebf73adabad1e8e26 P 85ce884025924a8591d29a026ec74b9e [term 1 FOLLOWER]: Refusing update from remote peer d6b88b1a75084198976f02d157d598b8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:14:32.270373 13627 raft_consensus.cc:1261] T 70995a7bad84452ebf73adabad1e8e26 P 2c609aa9e1ba44adae1501b81d613ca6 [term 1 FOLLOWER]: Refusing update from remote peer d6b88b1a75084198976f02d157d598b8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:14:32.271240 13835 consensus_queue.cc:1035] T 70995a7bad84452ebf73adabad1e8e26 P d6b88b1a75084198976f02d157d598b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85ce884025924a8591d29a026ec74b9e" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 45141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:14:32.271478 13832 consensus_queue.cc:1035] T 70995a7bad84452ebf73adabad1e8e26 P d6b88b1a75084198976f02d157d598b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c609aa9e1ba44adae1501b81d613ca6" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 36539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:14:32.280164 13840 mvcc.cc:205] Tried to move back new op lower bound from 7315339969610895360 to 7315339969443336192. Current Snapshot: MvccSnapshot[applied={T|T < 7315339969610895360}]
I20260805 23:14:32.281996 13839 mvcc.cc:205] Tried to move back new op lower bound from 7315339969610895360 to 7315339969443336192. Current Snapshot: MvccSnapshot[applied={T|T < 7315339969610895360}]
I20260805 23:14:32.291810 13342 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35478:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:14:32.292006 13342 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35478:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:14:32.294932 13342 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3ba684461f1f4ad0a79c797f551b3aee: Sending DeleteTablet for 3 replicas of tablet 70995a7bad84452ebf73adabad1e8e26
I20260805 23:14:32.295926 13741 tablet_service.cc:1514] Processing DeleteTablet for tablet 70995a7bad84452ebf73adabad1e8e26 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:14:32 UTC) from {username='kudu', principal='kudu/127.25.92.254@KRBTEST.COM'} at 127.0.0.1:43040
I20260805 23:14:32.295926 13605 tablet_service.cc:1514] Processing DeleteTablet for tablet 70995a7bad84452ebf73adabad1e8e26 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:14:32 UTC) from {username='kudu', principal='kudu/127.25.92.254@KRBTEST.COM'} at 127.0.0.1:42820
I20260805 23:14:32.296532 13469 tablet_service.cc:1514] Processing DeleteTablet for tablet 70995a7bad84452ebf73adabad1e8e26 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:14:32 UTC) from {username='kudu', principal='kudu/127.25.92.254@KRBTEST.COM'} at 127.0.0.1:47198
I20260805 23:14:32.296677 13849 tablet_replica.cc:330] T 70995a7bad84452ebf73adabad1e8e26 P 2c609aa9e1ba44adae1501b81d613ca6: stopping tablet replica
I20260805 23:14:32.297005 13850 tablet_replica.cc:330] T 70995a7bad84452ebf73adabad1e8e26 P 85ce884025924a8591d29a026ec74b9e: stopping tablet replica
I20260805 23:14:32.297060 13849 raft_consensus.cc:2229] T 70995a7bad84452ebf73adabad1e8e26 P 2c609aa9e1ba44adae1501b81d613ca6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 23:14:32.297286 13850 raft_consensus.cc:2229] T 70995a7bad84452ebf73adabad1e8e26 P 85ce884025924a8591d29a026ec74b9e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 23:14:32.297595 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 13406
I20260805 23:14:32.299371 13848 tablet_replica.cc:330] T 70995a7bad84452ebf73adabad1e8e26 P d6b88b1a75084198976f02d157d598b8: stopping tablet replica
I20260805 23:14:32.299751 13848 raft_consensus.cc:2229] T 70995a7bad84452ebf73adabad1e8e26 P d6b88b1a75084198976f02d157d598b8 [term 1 LEADER]: Raft consensus shutting down.
I20260805 23:14:32.297463 13849 raft_consensus.cc:2258] T 70995a7bad84452ebf73adabad1e8e26 P 2c609aa9e1ba44adae1501b81d613ca6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:14:32.300328 13848 raft_consensus.cc:2258] T 70995a7bad84452ebf73adabad1e8e26 P d6b88b1a75084198976f02d157d598b8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:14:32.302356 13849 ts_tablet_manager.cc:1905] T 70995a7bad84452ebf73adabad1e8e26 P 2c609aa9e1ba44adae1501b81d613ca6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:14:32.302693 13848 ts_tablet_manager.cc:1905] T 70995a7bad84452ebf73adabad1e8e26 P d6b88b1a75084198976f02d157d598b8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:14:32.305845 13848 ts_tablet_manager.cc:1918] T 70995a7bad84452ebf73adabad1e8e26 P d6b88b1a75084198976f02d157d598b8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 23:14:32.305961 13848 log.cc:1196] T 70995a7bad84452ebf73adabad1e8e26 P d6b88b1a75084198976f02d157d598b8: Deleting WAL directory at /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971406251336-25971-0/minicluster-data/ts-2/wal/wals/70995a7bad84452ebf73adabad1e8e26
I20260805 23:14:32.306286 13849 ts_tablet_manager.cc:1918] T 70995a7bad84452ebf73adabad1e8e26 P 2c609aa9e1ba44adae1501b81d613ca6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 23:14:32.306391 13849 log.cc:1196] T 70995a7bad84452ebf73adabad1e8e26 P 2c609aa9e1ba44adae1501b81d613ca6: Deleting WAL directory at /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971406251336-25971-0/minicluster-data/ts-1/wal/wals/70995a7bad84452ebf73adabad1e8e26
I20260805 23:14:32.306604 13848 ts_tablet_manager.cc:1939] T 70995a7bad84452ebf73adabad1e8e26 P d6b88b1a75084198976f02d157d598b8: Deleting consensus metadata
I20260805 23:14:32.306761 13849 ts_tablet_manager.cc:1939] T 70995a7bad84452ebf73adabad1e8e26 P 2c609aa9e1ba44adae1501b81d613ca6: Deleting consensus metadata
I20260805 23:14:32.308178 13325 catalog_manager.cc:4841] TS d6b88b1a75084198976f02d157d598b8 (127.25.92.195:42799): tablet 70995a7bad84452ebf73adabad1e8e26 (table test-table [id=45c111484ade465f88a68c38d83dbce5]) successfully deleted
I20260805 23:14:32.308229 13328 catalog_manager.cc:4841] TS 2c609aa9e1ba44adae1501b81d613ca6 (127.25.92.194:36539): tablet 70995a7bad84452ebf73adabad1e8e26 (table test-table [id=45c111484ade465f88a68c38d83dbce5]) successfully deleted
W20260805 23:14:32.311219 13327 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.25.92.193:45141 (error 108)
I20260805 23:14:32.312260 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 13542
I20260805 23:14:32.312494 13820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:32.312121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.193:45141 (local address 127.0.0.1:47206)
0805 23:14:32.312291 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:32.312383 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.193:45141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260805 23:14:32.313405 13327 catalog_manager.cc:4584] TS 85ce884025924a8591d29a026ec74b9e (127.25.92.193:45141): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 70995a7bad84452ebf73adabad1e8e26: Network error: Client connection negotiation failed: client connection to 127.25.92.193:45141: connect: Connection refused (error 111)
I20260805 23:14:32.321547 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 13678
I20260805 23:14:32.327762 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 13308
2026-08-05T23:14:32Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2687 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-05T23:14:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:14:32Z Disabled control of system clock
I20260805 23:14:32.361944 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971406251336-25971-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971406251336-25971-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971406251336-25971-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.254:37801
--webserver_interface=127.25.92.254
--webserver_port=0
--builtin_ntp_servers=127.25.92.212:39755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.92.254:37801
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260805 23:14:32.488400 13855 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:14:32.488698 13855 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:14:32.488736 13855 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:14:32.492846 13855 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:14:32.492921 13855 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:14:32.492936 13855 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:14:32.492954 13855 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:14:32.492969 13855 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:14:32.497928 13855 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:39755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971406251336-25971-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971406251336-25971-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.92.254:37801
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.254:37801
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.92.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13855
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971406251336-25971-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:14:32.499121 13855 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:14:32.500473 13855 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:14:32.509877 13863 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:14:32.509903 13861 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:14:32.509883 13860 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:14:32.510149 13855 server_base.cc:1034] running on GCE node
I20260805 23:14:32.510815 13855 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:14:32.512010 13855 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:14:32.513230 13855 hybrid_clock.cc:648] HybridClock initialized: now 1785971672513200 us; error 54 us; skew 500 ppm
I20260805 23:14:32.515537 13855 webserver.cc:460] Webserver started at http://127.25.92.254:42079/ using document root <none> and password file <none>
I20260805 23:14:32.516211 13855 fs_manager.cc:359] Metadata directory not provided
I20260805 23:14:32.516317 13855 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:14:32.516574 13855 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:14:32.518952 13855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971406251336-25971-0/minicluster-data/master-0/data/instance:
uuid: "9fbb8e78ae2e4b19ab291f88563a7643"
format_stamp: "Formatted at 2026-08-05 23:14:32 on dist-test-slave-ss4n"
server_key: "5b4aa75e785cad333dbed3fd6049b0f0"
server_key_iv: "90f95a6a1acb648266496aebcaf59cd5"
server_key_version: "encryptionkey@0"
I20260805 23:14:32.519562 13855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971406251336-25971-0/minicluster-data/master-0/wal/instance:
uuid: "9fbb8e78ae2e4b19ab291f88563a7643"
format_stamp: "Formatted at 2026-08-05 23:14:32 on dist-test-slave-ss4n"
server_key: "5b4aa75e785cad333dbed3fd6049b0f0"
server_key_iv: "90f95a6a1acb648266496aebcaf59cd5"
server_key_version: "encryptionkey@0"
I20260805 23:14:32.523693 13855 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260805 23:14:32.526607 13870 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:14:32.528048 13855 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:14:32.528221 13855 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971406251336-25971-0/minicluster-data/master-0/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971406251336-25971-0/minicluster-data/master-0/wal
uuid: "9fbb8e78ae2e4b19ab291f88563a7643"
format_stamp: "Formatted at 2026-08-05 23:14:32 on dist-test-slave-ss4n"
server_key: "5b4aa75e785cad333dbed3fd6049b0f0"
server_key_iv: "90f95a6a1acb648266496aebcaf59cd5"
server_key_version: "encryptionkey@0"
I20260805 23:14:32.528362 13855 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971406251336-25971-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971406251336-25971-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971406251336-25971-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:14:32.548842 13855 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:14:32.549875 13855 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:14:32.550171 13855 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:14:32.558564 13855 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.254:37801
I20260805 23:14:32.558597 13924 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.254:37801 every 8 connection(s)
I20260805 23:14:32.559664 13855 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
I20260805 23:14:32.563043 13925 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:14:32.569259 13925 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9fbb8e78ae2e4b19ab291f88563a7643: Bootstrap starting.
I20260805 23:14:32.569664 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 13855
I20260805 23:14:32.569803 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971406251336-25971-0/minicluster-data/master-0/wal/instance
I20260805 23:14:32.570106 25971 external_mini_cluster.cc:1428] Setting key 71608d74527687191794f9d74a639ada
I20260805 23:14:32.572366 13925 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9fbb8e78ae2e4b19ab291f88563a7643: Neither blocks nor log segments found. Creating new log.
I20260805 23:14:32.573163 13925 log.cc:824] T 00000000000000000000000000000000 P 9fbb8e78ae2e4b19ab291f88563a7643: Log is configured to *not* fsync() on all Append() calls
I20260805 23:14:32.575296 13925 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9fbb8e78ae2e4b19ab291f88563a7643: No bootstrap required, opened a new log
I20260805 23:14:32.579197 13925 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9fbb8e78ae2e4b19ab291f88563a7643 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fbb8e78ae2e4b19ab291f88563a7643" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 37801 } }
I20260805 23:14:32.579473 13925 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9fbb8e78ae2e4b19ab291f88563a7643 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:14:32.579561 13925 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9fbb8e78ae2e4b19ab291f88563a7643 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9fbb8e78ae2e4b19ab291f88563a7643, State: Initialized, Role: FOLLOWER
I20260805 23:14:32.580122 13925 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9fbb8e78ae2e4b19ab291f88563a7643 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fbb8e78ae2e4b19ab291f88563a7643" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 37801 } }
I20260805 23:14:32.580299 13925 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9fbb8e78ae2e4b19ab291f88563a7643 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:14:32.580392 13925 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9fbb8e78ae2e4b19ab291f88563a7643 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:14:32.580495 13925 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9fbb8e78ae2e4b19ab291f88563a7643 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:14:32.581161 13928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:32.571778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56342 (local address 127.25.92.254:37801)
0805 23:14:32.572574 (+ 796us) server_negotiation.cc:206] Beginning negotiation
0805 23:14:32.572615 (+ 41us) server_negotiation.cc:399] Waiting for connection header
0805 23:14:32.572656 (+ 41us) server_negotiation.cc:407] Connection header received
0805 23:14:32.573453 (+ 797us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:14:32.573539 (+ 86us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:14:32.573913 (+ 374us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:14:32.574293 (+ 380us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:14:32.574770 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:32.575915 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:32.576913 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:32.577529 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:32.577736 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:32.578903 (+ 1167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:14:32.578908 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:14:32.578924 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:14:32.579369 (+ 445us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:14:32.579446 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:14:32.579468 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:14:32.579603 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0805 23:14:32.579891 (+ 288us) server_negotiation.cc:299] Negotiation successful
0805 23:14:32.580172 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":144,"threads_started":1}
I20260805 23:14:32.581702 13925 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9fbb8e78ae2e4b19ab291f88563a7643 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fbb8e78ae2e4b19ab291f88563a7643" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 37801 } }
I20260805 23:14:32.582072 13925 leader_election.cc:302] T 00000000000000000000000000000000 P 9fbb8e78ae2e4b19ab291f88563a7643 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9fbb8e78ae2e4b19ab291f88563a7643; no voters:
I20260805 23:14:32.582479 13925 leader_election.cc:288] T 00000000000000000000000000000000 P 9fbb8e78ae2e4b19ab291f88563a7643 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 23:14:32.582556 13930 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9fbb8e78ae2e4b19ab291f88563a7643 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:14:32.582754 13930 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9fbb8e78ae2e4b19ab291f88563a7643 [term 1 LEADER]: Becoming Leader. State: Replica: 9fbb8e78ae2e4b19ab291f88563a7643, State: Running, Role: LEADER
I20260805 23:14:32.583060 13930 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9fbb8e78ae2e4b19ab291f88563a7643 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fbb8e78ae2e4b19ab291f88563a7643" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 37801 } }
I20260805 23:14:32.583815 13925 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9fbb8e78ae2e4b19ab291f88563a7643 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:14:32.584831 13931 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9fbb8e78ae2e4b19ab291f88563a7643 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9fbb8e78ae2e4b19ab291f88563a7643" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fbb8e78ae2e4b19ab291f88563a7643" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 37801 } } }
I20260805 23:14:32.584872 13932 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9fbb8e78ae2e4b19ab291f88563a7643 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9fbb8e78ae2e4b19ab291f88563a7643. Latest consensus state: current_term: 1 leader_uuid: "9fbb8e78ae2e4b19ab291f88563a7643" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fbb8e78ae2e4b19ab291f88563a7643" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 37801 } } }
I20260805 23:14:32.585103 13932 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9fbb8e78ae2e4b19ab291f88563a7643 [sys.catalog]: This master's current role is: LEADER
I20260805 23:14:32.585139 13931 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9fbb8e78ae2e4b19ab291f88563a7643 [sys.catalog]: This master's current role is: LEADER
I20260805 23:14:32.586018 13939 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:14:32.588815 13939 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:14:32.595148 13939 catalog_manager.cc:1347] Generated new cluster ID: 46b33c6a9317414780ca645244a34884
I20260805 23:14:32.595242 13939 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:14:32.606067 13939 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:14:32.607508 13939 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:14:32.622118 13939 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9fbb8e78ae2e4b19ab291f88563a7643: Generated new TSK 0
I20260805 23:14:32.623179 13939 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260805 23:14:32.626657 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971406251336-25971-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971406251336-25971-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971406251336-25971-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971406251336-25971-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.92.193:0
--local_ip_for_outbound_sockets=127.25.92.193
--webserver_interface=127.25.92.193
--webserver_port=0
--tserver_master_addrs=127.25.92.254:37801
--builtin_ntp_servers=127.25.92.212:39755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260805 23:14:32.754989 13950 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:14:32.755306 13950 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:14:32.755347 13950 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:14:32.759706 13950 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:14:32.759789 13950 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:14:32.759850 13950 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.92.193
I20260805 23:14:32.765183 13950 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:39755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971406251336-25971-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971406251336-25971-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.92.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971406251336-25971-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.92.193
--webserver_port=0
--tserver_master_addrs=127.25.92.254:37801
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13950
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.92.193
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971406251336-25971-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:14:32.766420 13950 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:14:32.767907 13950 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:14:32.778880 13955 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:14:32.778954 13950 server_base.cc:1034] running on GCE node
W20260805 23:14:32.778874 13958 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:14:32.778875 13956 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:14:32.779588 13950 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:14:32.780323 13950 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:14:32.781626 13950 hybrid_clock.cc:648] HybridClock initialized: now 1785971672781592 us; error 46 us; skew 500 ppm
I20260805 23:14:32.783926 13950 webserver.cc:460] Webserver started at http://127.25.92.193:41387/ using document root <none> and password file <none>
I20260805 23:14:32.784628 13950 fs_manager.cc:359] Metadata directory not provided
I20260805 23:14:32.784739 13950 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:14:32.784972 13950 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:14:32.787472 13950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971406251336-25971-0/minicluster-data/ts-0/data/instance:
uuid: "65f54374bb744e9fb4d8ac7b78eb0dcd"
format_stamp: "Formatted at 2026-08-05 23:14:32 on dist-test-slave-ss4n"
server_key: "3c4850bb5e7eba194819df32e705ce75"
server_key_iv: "2c56957057012126f23b30c611e5fd9c"
server_key_version: "encryptionkey@0"
I20260805 23:14:32.788097 13950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971406251336-25971-0/minicluster-data/ts-0/wal/instance:
uuid: "65f54374bb744e9fb4d8ac7b78eb0dcd"
format_stamp: "Formatted at 2026-08-05 23:14:32 on dist-test-slave-ss4n"
server_key: "3c4850bb5e7eba194819df32e705ce75"
server_key_iv: "2c56957057012126f23b30c611e5fd9c"
server_key_version: "encryptionkey@0"
I20260805 23:14:32.792160 13950 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260805 23:14:32.794962 13964 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:14:32.796360 13950 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260805 23:14:32.796525 13950 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971406251336-25971-0/minicluster-data/ts-0/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971406251336-25971-0/minicluster-data/ts-0/wal
uuid: "65f54374bb744e9fb4d8ac7b78eb0dcd"
format_stamp: "Formatted at 2026-08-05 23:14:32 on dist-test-slave-ss4n"
server_key: "3c4850bb5e7eba194819df32e705ce75"
server_key_iv: "2c56957057012126f23b30c611e5fd9c"
server_key_version: "encryptionkey@0"
I20260805 23:14:32.796677 13950 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971406251336-25971-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971406251336-25971-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971406251336-25971-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:14:32.808583 13950 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:14:32.809367 13950 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:14:32.809681 13950 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:14:32.810343 13950 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:14:32.811370 13950 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:14:32.811447 13950 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:14:32.811522 13950 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:14:32.811550 13950 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:14:32.821247 13950 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.193:36589
I20260805 23:14:32.821306 14078 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.193:36589 every 8 connection(s)
I20260805 23:14:32.822337 13950 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971406251336-25971-0/minicluster-data/ts-0/data/info.pb
I20260805 23:14:32.824784 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 13950
I20260805 23:14:32.824928 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971406251336-25971-0/minicluster-data/ts-0/wal/instance
I20260805 23:14:32.825284 25971 external_mini_cluster.cc:1428] Setting key 16627a91745490336233f518cd2fe45f
I20260805 23:14:32.827556 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971406251336-25971-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971406251336-25971-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971406251336-25971-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971406251336-25971-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.92.194:0
--local_ip_for_outbound_sockets=127.25.92.194
--webserver_interface=127.25.92.194
--webserver_port=0
--tserver_master_addrs=127.25.92.254:37801
--builtin_ntp_servers=127.25.92.212:39755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260805 23:14:32.833423 13928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:32.824330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.193:52577 (local address 127.25.92.254:37801)
0805 23:14:32.824518 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0805 23:14:32.824521 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 23:14:32.826020 (+ 1499us) server_negotiation.cc:407] Connection header received
0805 23:14:32.827862 (+ 1842us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:14:32.827865 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:14:32.827921 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:14:32.828005 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:14:32.829010 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:32.829843 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:32.831145 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:32.831395 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:32.831523 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:32.832853 (+ 1330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:14:32.832859 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:14:32.832867 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:14:32.832881 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:14:32.832901 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:14:32.832909 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:14:32.832989 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0805 23:14:32.833188 (+ 199us) server_negotiation.cc:299] Negotiation successful
0805 23:14:32.833261 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260805 23:14:32.834424 14081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:32.824849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:37801 (local address 127.25.92.193:52577)
0805 23:14:32.825848 (+ 999us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:32.825910 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0805 23:14:32.827559 (+ 1649us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:14:32.828218 (+ 659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:14:32.828227 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:14:32.828320 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:14:32.828850 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:14:32.828863 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:14:32.830007 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:14:32.830011 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:14:32.830970 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:14:32.830982 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:14:32.831587 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:14:32.831592 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:14:32.831782 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:32.832553 (+ 771us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0805 23:14:32.832554 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:14:32.832600 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0805 23:14:32.832676 (+ 76us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:14:32.833011 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:14:32.833023 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:14:32.833029 (+ 6us) client_negotiation.cc:769] Sending connection context
0805 23:14:32.833184 (+ 155us) client_negotiation.cc:240] Negotiation successful
0805 23:14:32.833531 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":180,"threads_started":1}
I20260805 23:14:32.835902 14079 heartbeater.cc:347] Connected to a master server at 127.25.92.254:37801
I20260805 23:14:32.836189 14079 heartbeater.cc:464] Registering TS with master...
I20260805 23:14:32.836814 14079 heartbeater.cc:511] Master 127.25.92.254:37801 requested a full tablet report, sending...
I20260805 23:14:32.838704 13887 ts_manager.cc:194] Registered new tserver with Master: 65f54374bb744e9fb4d8ac7b78eb0dcd (127.25.92.193:36589)
I20260805 23:14:32.840358 13887 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.25.92.193:52577
W20260805 23:14:32.949388 14082 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:14:32.949831 14082 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:14:32.949874 14082 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:14:32.954262 14082 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:14:32.954335 14082 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:14:32.954396 14082 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.92.194
I20260805 23:14:32.959685 14082 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:39755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971406251336-25971-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971406251336-25971-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.92.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971406251336-25971-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.92.194
--webserver_port=0
--tserver_master_addrs=127.25.92.254:37801
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14082
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.92.194
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971406251336-25971-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:14:32.960834 14082 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:14:32.962569 14082 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:14:32.974071 14090 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:14:32.974188 14088 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:14:32.974207 14087 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:14:32.974287 14082 server_base.cc:1034] running on GCE node
I20260805 23:14:32.974875 14082 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:14:32.975579 14082 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:14:32.976843 14082 hybrid_clock.cc:648] HybridClock initialized: now 1785971672976821 us; error 39 us; skew 500 ppm
I20260805 23:14:32.979367 14082 webserver.cc:460] Webserver started at http://127.25.92.194:34899/ using document root <none> and password file <none>
I20260805 23:14:32.980322 14082 fs_manager.cc:359] Metadata directory not provided
I20260805 23:14:32.980460 14082 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:14:32.980760 14082 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:14:32.983111 14082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971406251336-25971-0/minicluster-data/ts-1/data/instance:
uuid: "28bad0fe80174a468e9a3ded48416aa3"
format_stamp: "Formatted at 2026-08-05 23:14:32 on dist-test-slave-ss4n"
server_key: "fcaf359c53d5669825fc1c09ce952770"
server_key_iv: "639857bfb7d09e71fb9613fdafc47aa0"
server_key_version: "encryptionkey@0"
I20260805 23:14:32.983739 14082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971406251336-25971-0/minicluster-data/ts-1/wal/instance:
uuid: "28bad0fe80174a468e9a3ded48416aa3"
format_stamp: "Formatted at 2026-08-05 23:14:32 on dist-test-slave-ss4n"
server_key: "fcaf359c53d5669825fc1c09ce952770"
server_key_iv: "639857bfb7d09e71fb9613fdafc47aa0"
server_key_version: "encryptionkey@0"
I20260805 23:14:32.988049 14082 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260805 23:14:32.991017 14096 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:14:32.992491 14082 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260805 23:14:32.992668 14082 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971406251336-25971-0/minicluster-data/ts-1/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971406251336-25971-0/minicluster-data/ts-1/wal
uuid: "28bad0fe80174a468e9a3ded48416aa3"
format_stamp: "Formatted at 2026-08-05 23:14:32 on dist-test-slave-ss4n"
server_key: "fcaf359c53d5669825fc1c09ce952770"
server_key_iv: "639857bfb7d09e71fb9613fdafc47aa0"
server_key_version: "encryptionkey@0"
I20260805 23:14:32.992810 14082 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971406251336-25971-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971406251336-25971-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971406251336-25971-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:14:33.026744 14082 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:14:33.027505 14082 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:14:33.027729 14082 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:14:33.028352 14082 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:14:33.029362 14082 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:14:33.029434 14082 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:14:33.029527 14082 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:14:33.029572 14082 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:14:33.039551 14082 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.194:42555
I20260805 23:14:33.039597 14212 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.194:42555 every 8 connection(s)
I20260805 23:14:33.040541 14082 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971406251336-25971-0/minicluster-data/ts-1/data/info.pb
I20260805 23:14:33.045670 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 14082
I20260805 23:14:33.045831 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971406251336-25971-0/minicluster-data/ts-1/wal/instance
I20260805 23:14:33.046139 25971 external_mini_cluster.cc:1428] Setting key d6851fb679ff4cb20fd63623e4bf0d5a
I20260805 23:14:33.048259 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971406251336-25971-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971406251336-25971-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971406251336-25971-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971406251336-25971-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.92.195:0
--local_ip_for_outbound_sockets=127.25.92.195
--webserver_interface=127.25.92.195
--webserver_port=0
--tserver_master_addrs=127.25.92.254:37801
--builtin_ntp_servers=127.25.92.212:39755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260805 23:14:33.051290 13928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:33.042861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.194:58311 (local address 127.25.92.254:37801)
0805 23:14:33.043030 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0805 23:14:33.043035 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:14:33.044272 (+ 1237us) server_negotiation.cc:407] Connection header received
0805 23:14:33.045842 (+ 1570us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:14:33.045845 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:14:33.045912 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:14:33.046023 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:14:33.047168 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:33.047814 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:33.049125 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:33.049355 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:33.049472 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:33.050755 (+ 1283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:14:33.050759 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:14:33.050761 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:14:33.050774 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:14:33.050795 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:14:33.050803 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:14:33.050883 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0805 23:14:33.051063 (+ 180us) server_negotiation.cc:299] Negotiation successful
0805 23:14:33.051137 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260805 23:14:33.052383 14215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:33.043327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:37801 (local address 127.25.92.194:58311)
0805 23:14:33.044084 (+ 757us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:33.044129 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0805 23:14:33.045592 (+ 1463us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:14:33.046245 (+ 653us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:14:33.046254 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:14:33.046365 (+ 111us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:14:33.047006 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:14:33.047023 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:14:33.047964 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:14:33.047967 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:14:33.048978 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:14:33.048988 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:14:33.049550 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:14:33.049556 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:14:33.049734 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:33.050432 (+ 698us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0805 23:14:33.050433 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:14:33.050488 (+ 55us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0805 23:14:33.050564 (+ 76us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:14:33.050885 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:14:33.050909 (+ 24us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:14:33.050920 (+ 11us) client_negotiation.cc:769] Sending connection context
0805 23:14:33.051059 (+ 139us) client_negotiation.cc:240] Negotiation successful
0805 23:14:33.051379 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":139,"threads_started":1}
I20260805 23:14:33.053732 14213 heartbeater.cc:347] Connected to a master server at 127.25.92.254:37801
I20260805 23:14:33.054008 14213 heartbeater.cc:464] Registering TS with master...
I20260805 23:14:33.054581 14213 heartbeater.cc:511] Master 127.25.92.254:37801 requested a full tablet report, sending...
I20260805 23:14:33.055814 13887 ts_manager.cc:194] Registered new tserver with Master: 28bad0fe80174a468e9a3ded48416aa3 (127.25.92.194:42555)
I20260805 23:14:33.056732 13887 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.25.92.194:58311
W20260805 23:14:33.174038 14216 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:14:33.174443 14216 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:14:33.174504 14216 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:14:33.178733 14216 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:14:33.178821 14216 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:14:33.178884 14216 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.92.195
I20260805 23:14:33.183977 14216 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:39755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971406251336-25971-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971406251336-25971-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.92.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971406251336-25971-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.92.195
--webserver_port=0
--tserver_master_addrs=127.25.92.254:37801
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14216
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.92.195
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971406251336-25971-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:14:33.185292 14216 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:14:33.186852 14216 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:14:33.197906 14222 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:14:33.197911 14221 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:14:33.197996 14216 server_base.cc:1034] running on GCE node
W20260805 23:14:33.197911 14224 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:14:33.198752 14216 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:14:33.199460 14216 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:14:33.200706 14216 hybrid_clock.cc:648] HybridClock initialized: now 1785971673200688 us; error 28 us; skew 500 ppm
I20260805 23:14:33.202893 14216 webserver.cc:460] Webserver started at http://127.25.92.195:44419/ using document root <none> and password file <none>
I20260805 23:14:33.203485 14216 fs_manager.cc:359] Metadata directory not provided
I20260805 23:14:33.203553 14216 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:14:33.203735 14216 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:14:33.206107 14216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971406251336-25971-0/minicluster-data/ts-2/data/instance:
uuid: "836c226145af4934b2c23b6ea6a8ed51"
format_stamp: "Formatted at 2026-08-05 23:14:33 on dist-test-slave-ss4n"
server_key: "1a35cd45dfb97e0a1fd1c8df557b16a8"
server_key_iv: "538b34dd8c9fe5b316b260b1e07d01d5"
server_key_version: "encryptionkey@0"
I20260805 23:14:33.206670 14216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971406251336-25971-0/minicluster-data/ts-2/wal/instance:
uuid: "836c226145af4934b2c23b6ea6a8ed51"
format_stamp: "Formatted at 2026-08-05 23:14:33 on dist-test-slave-ss4n"
server_key: "1a35cd45dfb97e0a1fd1c8df557b16a8"
server_key_iv: "538b34dd8c9fe5b316b260b1e07d01d5"
server_key_version: "encryptionkey@0"
I20260805 23:14:33.210752 14216 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260805 23:14:33.213557 14230 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:14:33.215216 14216 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:14:33.215332 14216 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971406251336-25971-0/minicluster-data/ts-2/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971406251336-25971-0/minicluster-data/ts-2/wal
uuid: "836c226145af4934b2c23b6ea6a8ed51"
format_stamp: "Formatted at 2026-08-05 23:14:33 on dist-test-slave-ss4n"
server_key: "1a35cd45dfb97e0a1fd1c8df557b16a8"
server_key_iv: "538b34dd8c9fe5b316b260b1e07d01d5"
server_key_version: "encryptionkey@0"
I20260805 23:14:33.215458 14216 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971406251336-25971-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971406251336-25971-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971406251336-25971-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:14:33.242983 14216 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:14:33.243885 14216 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:14:33.244073 14216 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:14:33.244673 14216 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:14:33.245764 14216 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:14:33.245824 14216 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:14:33.245867 14216 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:14:33.245883 14216 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:14:33.255551 14216 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.195:45193
I20260805 23:14:33.255577 14346 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.195:45193 every 8 connection(s)
I20260805 23:14:33.256645 14216 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971406251336-25971-0/minicluster-data/ts-2/data/info.pb
I20260805 23:14:33.266173 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 14216
I20260805 23:14:33.266345 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971406251336-25971-0/minicluster-data/ts-2/wal/instance
I20260805 23:14:33.266786 25971 external_mini_cluster.cc:1428] Setting key 301fe76ff593542035fbe2f57f513c82
I20260805 23:14:33.267810 13928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:33.258919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.195:55705 (local address 127.25.92.254:37801)
0805 23:14:33.259115 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0805 23:14:33.259120 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:14:33.260283 (+ 1163us) server_negotiation.cc:407] Connection header received
0805 23:14:33.261953 (+ 1670us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:14:33.261957 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:14:33.262016 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:14:33.262115 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:14:33.263227 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:33.263791 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:33.265127 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:33.265351 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:33.265449 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:33.267028 (+ 1579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:14:33.267032 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:14:33.267034 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:14:33.267049 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:14:33.267073 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:14:33.267081 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:14:33.267152 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0805 23:14:33.267564 (+ 412us) server_negotiation.cc:299] Negotiation successful
0805 23:14:33.267651 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260805 23:14:33.268880 14349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:33.259335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:37801 (local address 127.25.92.195:55705)
0805 23:14:33.260093 (+ 758us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:33.260143 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0805 23:14:33.261682 (+ 1539us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:14:33.262348 (+ 666us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:14:33.262356 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:14:33.262446 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:14:33.263041 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:14:33.263054 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:14:33.263954 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:14:33.263957 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:14:33.264966 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:14:33.264978 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:14:33.265653 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:14:33.265657 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:14:33.265823 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:33.266611 (+ 788us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0805 23:14:33.266613 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:14:33.266678 (+ 65us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0805 23:14:33.266778 (+ 100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:14:33.267357 (+ 579us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:14:33.267366 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:14:33.267371 (+ 5us) client_negotiation.cc:769] Sending connection context
0805 23:14:33.267488 (+ 117us) client_negotiation.cc:240] Negotiation successful
0805 23:14:33.267854 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":132,"threads_started":1}
I20260805 23:14:33.270200 14347 heartbeater.cc:347] Connected to a master server at 127.25.92.254:37801
I20260805 23:14:33.270474 14347 heartbeater.cc:464] Registering TS with master...
I20260805 23:14:33.271126 14347 heartbeater.cc:511] Master 127.25.92.254:37801 requested a full tablet report, sending...
I20260805 23:14:33.272274 13887 ts_manager.cc:194] Registered new tserver with Master: 836c226145af4934b2c23b6ea6a8ed51 (127.25.92.195:45193)
I20260805 23:14:33.272881 13887 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.25.92.195:55705
I20260805 23:14:33.281852 25971 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260805 23:14:33.288733 13928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:33.284044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56354 (local address 127.25.92.254:37801)
0805 23:14:33.284211 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0805 23:14:33.284214 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 23:14:33.284364 (+ 150us) server_negotiation.cc:407] Connection header received
0805 23:14:33.284461 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:14:33.284464 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:14:33.284549 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:14:33.284715 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:14:33.284971 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:33.285807 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:33.286767 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:33.287001 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:33.287099 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:33.288102 (+ 1003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:14:33.288105 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:14:33.288107 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:14:33.288121 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:14:33.288143 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:14:33.288152 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:14:33.288219 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0805 23:14:33.288453 (+ 234us) server_negotiation.cc:299] Negotiation successful
0805 23:14:33.288593 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260805 23:14:33.292192 13887 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56354:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260805 23:14:33.295732 13887 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260805 23:14:33.312945 14362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:33.305607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.193:36589 (local address 127.0.0.1:38918)
0805 23:14:33.306831 (+ 1224us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:33.306888 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0805 23:14:33.307020 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:14:33.307892 (+ 872us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:14:33.307895 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:14:33.307923 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:14:33.308152 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:14:33.308159 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:14:33.309668 (+ 1509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:14:33.309674 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:14:33.311340 (+ 1666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:14:33.311352 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:14:33.312527 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:14:33.312531 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:14:33.312630 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:33.312656 (+ 26us) client_negotiation.cc:769] Sending connection context
0805 23:14:33.312715 (+ 59us) client_negotiation.cc:240] Negotiation successful
0805 23:14:33.312785 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1039,"mutex_wait_us":141,"spinlock_wait_cycles":1792,"thread_start_us":106,"threads_started":1}
I20260805 23:14:33.312970 14361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:33.306020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57972 (local address 127.25.92.195:45193)
0805 23:14:33.306528 (+ 508us) server_negotiation.cc:206] Beginning negotiation
0805 23:14:33.306545 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0805 23:14:33.307264 (+ 719us) server_negotiation.cc:407] Connection header received
0805 23:14:33.307506 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:14:33.307511 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:14:33.307702 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:14:33.307885 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:14:33.308453 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:33.309634 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:33.311513 (+ 1879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:33.312054 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:33.312195 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:33.312387 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0805 23:14:33.312703 (+ 316us) server_negotiation.cc:299] Negotiation successful
0805 23:14:33.312810 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":104,"threads_started":1}
I20260805 23:14:33.313416 14359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:33.305676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.195:45193 (local address 127.0.0.1:57972)
0805 23:14:33.307148 (+ 1472us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:33.307163 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0805 23:14:33.307347 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:14:33.307876 (+ 529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:14:33.307882 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:14:33.307919 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:14:33.308321 (+ 402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:14:33.308327 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:14:33.309753 (+ 1426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:14:33.309756 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:14:33.311345 (+ 1589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:14:33.311352 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:14:33.312450 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:14:33.312453 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:14:33.312546 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:33.312569 (+ 23us) client_negotiation.cc:769] Sending connection context
0805 23:14:33.313177 (+ 608us) client_negotiation.cc:240] Negotiation successful
0805 23:14:33.313269 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1397,"mutex_wait_us":173,"spinlock_wait_cycles":3968,"thread_start_us":124,"threads_started":1}
I20260805 23:14:33.313417 14360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:33.306010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38918 (local address 127.25.92.193:36589)
0805 23:14:33.307400 (+ 1390us) server_negotiation.cc:206] Beginning negotiation
0805 23:14:33.307409 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0805 23:14:33.307435 (+ 26us) server_negotiation.cc:407] Connection header received
0805 23:14:33.307527 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:14:33.307534 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:14:33.307728 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:14:33.308090 (+ 362us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:14:33.308237 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:33.309521 (+ 1284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:33.311487 (+ 1966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:33.312356 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:33.312883 (+ 527us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:33.313083 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0805 23:14:33.313144 (+ 61us) server_negotiation.cc:299] Negotiation successful
0805 23:14:33.313245 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1275,"thread_start_us":87,"threads_started":1}
I20260805 23:14:33.312798 14363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:33.306718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53410 (local address 127.25.92.194:42555)
0805 23:14:33.307210 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0805 23:14:33.307223 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0805 23:14:33.307553 (+ 330us) server_negotiation.cc:407] Connection header received
0805 23:14:33.307626 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:14:33.307631 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:14:33.307776 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:14:33.307913 (+ 137us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:14:33.308334 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:33.309409 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:33.311139 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:33.311984 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:33.312124 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:33.312363 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0805 23:14:33.312478 (+ 115us) server_negotiation.cc:299] Negotiation successful
0805 23:14:33.312604 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382,"thread_start_us":136,"threads_started":1}
I20260805 23:14:33.312768 14358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:33.306456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.194:42555 (local address 127.0.0.1:53410)
0805 23:14:33.307353 (+ 897us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:33.307366 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0805 23:14:33.307459 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:14:33.308083 (+ 624us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:14:33.308085 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:14:33.308101 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:14:33.308193 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:14:33.308200 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:14:33.309607 (+ 1407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:14:33.309614 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:14:33.310989 (+ 1375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:14:33.311001 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:14:33.312157 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:14:33.312164 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:14:33.312276 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:33.312316 (+ 40us) client_negotiation.cc:769] Sending connection context
0805 23:14:33.312438 (+ 122us) client_negotiation.cc:240] Negotiation successful
0805 23:14:33.312574 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":838,"mutex_wait_us":78,"thread_start_us":107,"threads_started":1}
I20260805 23:14:33.316949 14279 tablet_service.cc:1467] Processing CreateTablet for tablet 38f5e5cac6e44cc39a4484d96f16791b (DEFAULT_TABLE table=test-table [id=f92676b1d23f4ef286d92a3806908ca7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:14:33.316929 14144 tablet_service.cc:1467] Processing CreateTablet for tablet 38f5e5cac6e44cc39a4484d96f16791b (DEFAULT_TABLE table=test-table [id=f92676b1d23f4ef286d92a3806908ca7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:14:33.318300 14279 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38f5e5cac6e44cc39a4484d96f16791b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:14:33.318357 14144 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38f5e5cac6e44cc39a4484d96f16791b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:14:33.318843 14012 tablet_service.cc:1467] Processing CreateTablet for tablet 38f5e5cac6e44cc39a4484d96f16791b (DEFAULT_TABLE table=test-table [id=f92676b1d23f4ef286d92a3806908ca7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:14:33.319929 14012 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38f5e5cac6e44cc39a4484d96f16791b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:14:33.326426 14364 tablet_bootstrap.cc:492] T 38f5e5cac6e44cc39a4484d96f16791b P 65f54374bb744e9fb4d8ac7b78eb0dcd: Bootstrap starting.
I20260805 23:14:33.328097 14365 tablet_bootstrap.cc:492] T 38f5e5cac6e44cc39a4484d96f16791b P 28bad0fe80174a468e9a3ded48416aa3: Bootstrap starting.
I20260805 23:14:33.329445 14364 tablet_bootstrap.cc:654] T 38f5e5cac6e44cc39a4484d96f16791b P 65f54374bb744e9fb4d8ac7b78eb0dcd: Neither blocks nor log segments found. Creating new log.
I20260805 23:14:33.329991 14366 tablet_bootstrap.cc:492] T 38f5e5cac6e44cc39a4484d96f16791b P 836c226145af4934b2c23b6ea6a8ed51: Bootstrap starting.
I20260805 23:14:33.330343 14365 tablet_bootstrap.cc:654] T 38f5e5cac6e44cc39a4484d96f16791b P 28bad0fe80174a468e9a3ded48416aa3: Neither blocks nor log segments found. Creating new log.
I20260805 23:14:33.330443 14364 log.cc:824] T 38f5e5cac6e44cc39a4484d96f16791b P 65f54374bb744e9fb4d8ac7b78eb0dcd: Log is configured to *not* fsync() on all Append() calls
I20260805 23:14:33.331171 14365 log.cc:824] T 38f5e5cac6e44cc39a4484d96f16791b P 28bad0fe80174a468e9a3ded48416aa3: Log is configured to *not* fsync() on all Append() calls
I20260805 23:14:33.332137 14366 tablet_bootstrap.cc:654] T 38f5e5cac6e44cc39a4484d96f16791b P 836c226145af4934b2c23b6ea6a8ed51: Neither blocks nor log segments found. Creating new log.
I20260805 23:14:33.332823 14364 tablet_bootstrap.cc:492] T 38f5e5cac6e44cc39a4484d96f16791b P 65f54374bb744e9fb4d8ac7b78eb0dcd: No bootstrap required, opened a new log
I20260805 23:14:33.332940 14366 log.cc:824] T 38f5e5cac6e44cc39a4484d96f16791b P 836c226145af4934b2c23b6ea6a8ed51: Log is configured to *not* fsync() on all Append() calls
I20260805 23:14:33.333114 14364 ts_tablet_manager.cc:1397] T 38f5e5cac6e44cc39a4484d96f16791b P 65f54374bb744e9fb4d8ac7b78eb0dcd: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260805 23:14:33.333364 14365 tablet_bootstrap.cc:492] T 38f5e5cac6e44cc39a4484d96f16791b P 28bad0fe80174a468e9a3ded48416aa3: No bootstrap required, opened a new log
I20260805 23:14:33.333592 14365 ts_tablet_manager.cc:1397] T 38f5e5cac6e44cc39a4484d96f16791b P 28bad0fe80174a468e9a3ded48416aa3: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260805 23:14:33.335210 14366 tablet_bootstrap.cc:492] T 38f5e5cac6e44cc39a4484d96f16791b P 836c226145af4934b2c23b6ea6a8ed51: No bootstrap required, opened a new log
I20260805 23:14:33.335405 14366 ts_tablet_manager.cc:1397] T 38f5e5cac6e44cc39a4484d96f16791b P 836c226145af4934b2c23b6ea6a8ed51: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260805 23:14:33.337172 14364 raft_consensus.cc:348] T 38f5e5cac6e44cc39a4484d96f16791b P 65f54374bb744e9fb4d8ac7b78eb0dcd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65f54374bb744e9fb4d8ac7b78eb0dcd" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 36589 } } peers { permanent_uuid: "836c226145af4934b2c23b6ea6a8ed51" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 45193 } } peers { permanent_uuid: "28bad0fe80174a468e9a3ded48416aa3" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 42555 } }
I20260805 23:14:33.337450 14364 raft_consensus.cc:374] T 38f5e5cac6e44cc39a4484d96f16791b P 65f54374bb744e9fb4d8ac7b78eb0dcd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:14:33.337535 14364 raft_consensus.cc:729] T 38f5e5cac6e44cc39a4484d96f16791b P 65f54374bb744e9fb4d8ac7b78eb0dcd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65f54374bb744e9fb4d8ac7b78eb0dcd, State: Initialized, Role: FOLLOWER
I20260805 23:14:33.337996 14364 consensus_queue.cc:260] T 38f5e5cac6e44cc39a4484d96f16791b P 65f54374bb744e9fb4d8ac7b78eb0dcd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65f54374bb744e9fb4d8ac7b78eb0dcd" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 36589 } } peers { permanent_uuid: "836c226145af4934b2c23b6ea6a8ed51" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 45193 } } peers { permanent_uuid: "28bad0fe80174a468e9a3ded48416aa3" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 42555 } }
I20260805 23:14:33.338340 14365 raft_consensus.cc:348] T 38f5e5cac6e44cc39a4484d96f16791b P 28bad0fe80174a468e9a3ded48416aa3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65f54374bb744e9fb4d8ac7b78eb0dcd" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 36589 } } peers { permanent_uuid: "836c226145af4934b2c23b6ea6a8ed51" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 45193 } } peers { permanent_uuid: "28bad0fe80174a468e9a3ded48416aa3" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 42555 } }
I20260805 23:14:33.338629 14365 raft_consensus.cc:374] T 38f5e5cac6e44cc39a4484d96f16791b P 28bad0fe80174a468e9a3ded48416aa3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:14:33.338683 14079 heartbeater.cc:503] Master 127.25.92.254:37801 was elected leader, sending a full tablet report...
I20260805 23:14:33.338690 14365 raft_consensus.cc:729] T 38f5e5cac6e44cc39a4484d96f16791b P 28bad0fe80174a468e9a3ded48416aa3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28bad0fe80174a468e9a3ded48416aa3, State: Initialized, Role: FOLLOWER
I20260805 23:14:33.339087 14364 ts_tablet_manager.cc:1428] T 38f5e5cac6e44cc39a4484d96f16791b P 65f54374bb744e9fb4d8ac7b78eb0dcd: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260805 23:14:33.339164 14365 consensus_queue.cc:260] T 38f5e5cac6e44cc39a4484d96f16791b P 28bad0fe80174a468e9a3ded48416aa3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65f54374bb744e9fb4d8ac7b78eb0dcd" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 36589 } } peers { permanent_uuid: "836c226145af4934b2c23b6ea6a8ed51" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 45193 } } peers { permanent_uuid: "28bad0fe80174a468e9a3ded48416aa3" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 42555 } }
I20260805 23:14:33.339988 14213 heartbeater.cc:503] Master 127.25.92.254:37801 was elected leader, sending a full tablet report...
I20260805 23:14:33.340226 14365 ts_tablet_manager.cc:1428] T 38f5e5cac6e44cc39a4484d96f16791b P 28bad0fe80174a468e9a3ded48416aa3: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260805 23:14:33.340655 14366 raft_consensus.cc:348] T 38f5e5cac6e44cc39a4484d96f16791b P 836c226145af4934b2c23b6ea6a8ed51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65f54374bb744e9fb4d8ac7b78eb0dcd" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 36589 } } peers { permanent_uuid: "836c226145af4934b2c23b6ea6a8ed51" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 45193 } } peers { permanent_uuid: "28bad0fe80174a468e9a3ded48416aa3" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 42555 } }
I20260805 23:14:33.340885 14366 raft_consensus.cc:374] T 38f5e5cac6e44cc39a4484d96f16791b P 836c226145af4934b2c23b6ea6a8ed51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:14:33.340945 14366 raft_consensus.cc:729] T 38f5e5cac6e44cc39a4484d96f16791b P 836c226145af4934b2c23b6ea6a8ed51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 836c226145af4934b2c23b6ea6a8ed51, State: Initialized, Role: FOLLOWER
I20260805 23:14:33.341374 14366 consensus_queue.cc:260] T 38f5e5cac6e44cc39a4484d96f16791b P 836c226145af4934b2c23b6ea6a8ed51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65f54374bb744e9fb4d8ac7b78eb0dcd" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 36589 } } peers { permanent_uuid: "836c226145af4934b2c23b6ea6a8ed51" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 45193 } } peers { permanent_uuid: "28bad0fe80174a468e9a3ded48416aa3" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 42555 } }
I20260805 23:14:33.342321 14366 ts_tablet_manager.cc:1428] T 38f5e5cac6e44cc39a4484d96f16791b P 836c226145af4934b2c23b6ea6a8ed51: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260805 23:14:33.342691 14347 heartbeater.cc:503] Master 127.25.92.254:37801 was elected leader, sending a full tablet report...
I20260805 23:14:33.343626 14371 raft_consensus.cc:482] T 38f5e5cac6e44cc39a4484d96f16791b P 28bad0fe80174a468e9a3ded48416aa3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 23:14:33.343802 14371 raft_consensus.cc:504] T 38f5e5cac6e44cc39a4484d96f16791b P 28bad0fe80174a468e9a3ded48416aa3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65f54374bb744e9fb4d8ac7b78eb0dcd" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 36589 } } peers { permanent_uuid: "836c226145af4934b2c23b6ea6a8ed51" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 45193 } } peers { permanent_uuid: "28bad0fe80174a468e9a3ded48416aa3" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 42555 } }
I20260805 23:14:33.345036 14371 leader_election.cc:288] T 38f5e5cac6e44cc39a4484d96f16791b P 28bad0fe80174a468e9a3ded48416aa3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 65f54374bb744e9fb4d8ac7b78eb0dcd (127.25.92.193:36589), 836c226145af4934b2c23b6ea6a8ed51 (127.25.92.195:45193)
I20260805 23:14:33.350171 14361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:33.345814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.194:51435 (local address 127.25.92.195:45193)
0805 23:14:33.345951 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0805 23:14:33.345956 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:14:33.345970 (+ 14us) server_negotiation.cc:407] Connection header received
0805 23:14:33.346033 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:14:33.346036 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:14:33.346089 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:14:33.346170 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:14:33.346598 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:33.347572 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:33.348725 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:33.349373 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:33.349550 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:33.349617 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0805 23:14:33.349958 (+ 341us) server_negotiation.cc:299] Negotiation successful
0805 23:14:33.350032 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260805 23:14:33.350171 14360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:33.345944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.92.194:35425 (local address 127.25.92.193:36589)
0805 23:14:33.346058 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0805 23:14:33.346062 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:14:33.346267 (+ 205us) server_negotiation.cc:407] Connection header received
0805 23:14:33.346483 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:14:33.346487 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:14:33.346539 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:14:33.346589 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:14:33.347153 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:33.347745 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:33.348887 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:33.349558 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:33.349686 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:33.349731 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0805 23:14:33.349955 (+ 224us) server_negotiation.cc:299] Negotiation successful
0805 23:14:33.350032 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260805 23:14:33.350246 14215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:33.345382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.195:45193 (local address 127.25.92.194:51435)
0805 23:14:33.345563 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:33.345583 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0805 23:14:33.345672 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:14:33.346307 (+ 635us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:14:33.346310 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:14:33.346339 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:14:33.346462 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:14:33.346470 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:14:33.347698 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:14:33.347703 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:14:33.348573 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:14:33.348583 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:14:33.349645 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:14:33.349650 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:14:33.349774 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:33.349843 (+ 69us) client_negotiation.cc:769] Sending connection context
0805 23:14:33.349928 (+ 85us) client_negotiation.cc:240] Negotiation successful
0805 23:14:33.350074 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34,"mutex_wait_us":87,"spinlock_wait_cycles":3968}
I20260805 23:14:33.350718 14373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:33.345844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.193:36589 (local address 127.25.92.194:35425)
0805 23:14:33.346177 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:33.346192 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0805 23:14:33.346355 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:14:33.346823 (+ 468us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:14:33.346826 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:14:33.346849 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:14:33.347013 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:14:33.347020 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:14:33.347866 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:14:33.347868 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:14:33.348761 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:14:33.348771 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:14:33.349682 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:14:33.349685 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:14:33.349778 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:33.349855 (+ 77us) client_negotiation.cc:769] Sending connection context
0805 23:14:33.349927 (+ 72us) client_negotiation.cc:240] Negotiation successful
0805 23:14:33.350088 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"mutex_wait_us":108,"spinlock_wait_cycles":768,"thread_start_us":130,"threads_started":1}
I20260805 23:14:33.351040 14300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38f5e5cac6e44cc39a4484d96f16791b" candidate_uuid: "28bad0fe80174a468e9a3ded48416aa3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "836c226145af4934b2c23b6ea6a8ed51" is_pre_election: true
I20260805 23:14:33.351328 14033 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38f5e5cac6e44cc39a4484d96f16791b" candidate_uuid: "28bad0fe80174a468e9a3ded48416aa3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65f54374bb744e9fb4d8ac7b78eb0dcd" is_pre_election: true
I20260805 23:14:33.351383 14300 raft_consensus.cc:2454] T 38f5e5cac6e44cc39a4484d96f16791b P 836c226145af4934b2c23b6ea6a8ed51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bad0fe80174a468e9a3ded48416aa3 in term 0.
I20260805 23:14:33.351598 14033 raft_consensus.cc:2454] T 38f5e5cac6e44cc39a4484d96f16791b P 65f54374bb744e9fb4d8ac7b78eb0dcd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28bad0fe80174a468e9a3ded48416aa3 in term 0.
I20260805 23:14:33.351989 14098 leader_election.cc:302] T 38f5e5cac6e44cc39a4484d96f16791b P 28bad0fe80174a468e9a3ded48416aa3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bad0fe80174a468e9a3ded48416aa3, 836c226145af4934b2c23b6ea6a8ed51; no voters:
I20260805 23:14:33.352272 14371 raft_consensus.cc:2781] T 38f5e5cac6e44cc39a4484d96f16791b P 28bad0fe80174a468e9a3ded48416aa3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260805 23:14:33.352384 14371 raft_consensus.cc:482] T 38f5e5cac6e44cc39a4484d96f16791b P 28bad0fe80174a468e9a3ded48416aa3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260805 23:14:33.352442 14371 raft_consensus.cc:3037] T 38f5e5cac6e44cc39a4484d96f16791b P 28bad0fe80174a468e9a3ded48416aa3 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:14:33.353653 14371 raft_consensus.cc:504] T 38f5e5cac6e44cc39a4484d96f16791b P 28bad0fe80174a468e9a3ded48416aa3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65f54374bb744e9fb4d8ac7b78eb0dcd" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 36589 } } peers { permanent_uuid: "836c226145af4934b2c23b6ea6a8ed51" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 45193 } } peers { permanent_uuid: "28bad0fe80174a468e9a3ded48416aa3" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 42555 } }
I20260805 23:14:33.354027 14371 leader_election.cc:288] T 38f5e5cac6e44cc39a4484d96f16791b P 28bad0fe80174a468e9a3ded48416aa3 [CANDIDATE]: Term 1 election: Requested vote from peers 65f54374bb744e9fb4d8ac7b78eb0dcd (127.25.92.193:36589), 836c226145af4934b2c23b6ea6a8ed51 (127.25.92.195:45193)
I20260805 23:14:33.354460 14300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38f5e5cac6e44cc39a4484d96f16791b" candidate_uuid: "28bad0fe80174a468e9a3ded48416aa3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "836c226145af4934b2c23b6ea6a8ed51"
I20260805 23:14:33.354460 14033 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38f5e5cac6e44cc39a4484d96f16791b" candidate_uuid: "28bad0fe80174a468e9a3ded48416aa3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65f54374bb744e9fb4d8ac7b78eb0dcd"
I20260805 23:14:33.354583 14300 raft_consensus.cc:3037] T 38f5e5cac6e44cc39a4484d96f16791b P 836c226145af4934b2c23b6ea6a8ed51 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:14:33.354624 14033 raft_consensus.cc:3037] T 38f5e5cac6e44cc39a4484d96f16791b P 65f54374bb744e9fb4d8ac7b78eb0dcd [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:14:33.355686 14300 raft_consensus.cc:2454] T 38f5e5cac6e44cc39a4484d96f16791b P 836c226145af4934b2c23b6ea6a8ed51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bad0fe80174a468e9a3ded48416aa3 in term 1.
I20260805 23:14:33.356051 14033 raft_consensus.cc:2454] T 38f5e5cac6e44cc39a4484d96f16791b P 65f54374bb744e9fb4d8ac7b78eb0dcd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28bad0fe80174a468e9a3ded48416aa3 in term 1.
I20260805 23:14:33.356048 14098 leader_election.cc:302] T 38f5e5cac6e44cc39a4484d96f16791b P 28bad0fe80174a468e9a3ded48416aa3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bad0fe80174a468e9a3ded48416aa3, 836c226145af4934b2c23b6ea6a8ed51; no voters:
I20260805 23:14:33.356259 14371 raft_consensus.cc:2781] T 38f5e5cac6e44cc39a4484d96f16791b P 28bad0fe80174a468e9a3ded48416aa3 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:14:33.356554 14371 raft_consensus.cc:686] T 38f5e5cac6e44cc39a4484d96f16791b P 28bad0fe80174a468e9a3ded48416aa3 [term 1 LEADER]: Becoming Leader. State: Replica: 28bad0fe80174a468e9a3ded48416aa3, State: Running, Role: LEADER
I20260805 23:14:33.356966 14371 consensus_queue.cc:237] T 38f5e5cac6e44cc39a4484d96f16791b P 28bad0fe80174a468e9a3ded48416aa3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65f54374bb744e9fb4d8ac7b78eb0dcd" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 36589 } } peers { permanent_uuid: "836c226145af4934b2c23b6ea6a8ed51" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 45193 } } peers { permanent_uuid: "28bad0fe80174a468e9a3ded48416aa3" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 42555 } }
I20260805 23:14:33.360646 13884 catalog_manager.cc:5495] T 38f5e5cac6e44cc39a4484d96f16791b P 28bad0fe80174a468e9a3ded48416aa3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 28bad0fe80174a468e9a3ded48416aa3 (127.25.92.194). New cstate: current_term: 1 leader_uuid: "28bad0fe80174a468e9a3ded48416aa3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65f54374bb744e9fb4d8ac7b78eb0dcd" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 36589 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "836c226145af4934b2c23b6ea6a8ed51" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 45193 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bad0fe80174a468e9a3ded48416aa3" member_type: VOTER last_known_addr { host: "127.25.92.194" port: 42555 } health_report { overall_health: HEALTHY } } }
I20260805 23:14:33.378499 14363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:33.374811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53412 (local address 127.25.92.194:42555)
0805 23:14:33.374953 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0805 23:14:33.374961 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0805 23:14:33.374991 (+ 30us) server_negotiation.cc:407] Connection header received
0805 23:14:33.375085 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:14:33.375087 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:14:33.375145 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:14:33.375259 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 23:14:33.375526 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:33.376030 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:33.376889 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:33.377111 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:33.377219 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:33.377622 (+ 403us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 23:14:33.378075 (+ 453us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 23:14:33.378216 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0805 23:14:33.378276 (+ 60us) server_negotiation.cc:299] Negotiation successful
0805 23:14:33.378337 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260805 23:14:33.385006 14300 raft_consensus.cc:1261] T 38f5e5cac6e44cc39a4484d96f16791b P 836c226145af4934b2c23b6ea6a8ed51 [term 1 FOLLOWER]: Refusing update from remote peer 28bad0fe80174a468e9a3ded48416aa3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:14:33.385134 14033 raft_consensus.cc:1261] T 38f5e5cac6e44cc39a4484d96f16791b P 65f54374bb744e9fb4d8ac7b78eb0dcd [term 1 FOLLOWER]: Refusing update from remote peer 28bad0fe80174a468e9a3ded48416aa3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:14:33.385864 14371 consensus_queue.cc:1035] T 38f5e5cac6e44cc39a4484d96f16791b P 28bad0fe80174a468e9a3ded48416aa3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "836c226145af4934b2c23b6ea6a8ed51" member_type: VOTER last_known_addr { host: "127.25.92.195" port: 45193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:14:33.386137 14374 consensus_queue.cc:1035] T 38f5e5cac6e44cc39a4484d96f16791b P 28bad0fe80174a468e9a3ded48416aa3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "65f54374bb744e9fb4d8ac7b78eb0dcd" member_type: VOTER last_known_addr { host: "127.25.92.193" port: 36589 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:14:33.392864 14377 mvcc.cc:205] Tried to move back new op lower bound from 7315339974176067584 to 7315339974073520128. Current Snapshot: MvccSnapshot[applied={T|T < 7315339974176067584}]
I20260805 23:14:33.393854 14378 mvcc.cc:205] Tried to move back new op lower bound from 7315339974176067584 to 7315339974073520128. Current Snapshot: MvccSnapshot[applied={T|T < 7315339974176067584}]
I20260805 23:14:33.404675 13884 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:56354:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:14:33.404850 13884 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:56354:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:14:33.407694 13884 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9fbb8e78ae2e4b19ab291f88563a7643: Sending DeleteTablet for 3 replicas of tablet 38f5e5cac6e44cc39a4484d96f16791b
I20260805 23:14:33.408630 14279 tablet_service.cc:1514] Processing DeleteTablet for tablet 38f5e5cac6e44cc39a4484d96f16791b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:14:33 UTC) from {username='slave'} at 127.0.0.1:57972
I20260805 23:14:33.408633 14012 tablet_service.cc:1514] Processing DeleteTablet for tablet 38f5e5cac6e44cc39a4484d96f16791b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:14:33 UTC) from {username='slave'} at 127.0.0.1:38918
I20260805 23:14:33.408994 14144 tablet_service.cc:1514] Processing DeleteTablet for tablet 38f5e5cac6e44cc39a4484d96f16791b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:14:33 UTC) from {username='slave'} at 127.0.0.1:53410
I20260805 23:14:33.409238 14386 tablet_replica.cc:330] T 38f5e5cac6e44cc39a4484d96f16791b P 836c226145af4934b2c23b6ea6a8ed51: stopping tablet replica
I20260805 23:14:33.409454 14387 tablet_replica.cc:330] T 38f5e5cac6e44cc39a4484d96f16791b P 28bad0fe80174a468e9a3ded48416aa3: stopping tablet replica
I20260805 23:14:33.409652 14386 raft_consensus.cc:2229] T 38f5e5cac6e44cc39a4484d96f16791b P 836c226145af4934b2c23b6ea6a8ed51 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 23:14:33.409736 14387 raft_consensus.cc:2229] T 38f5e5cac6e44cc39a4484d96f16791b P 28bad0fe80174a468e9a3ded48416aa3 [term 1 LEADER]: Raft consensus shutting down.
I20260805 23:14:33.409989 14386 raft_consensus.cc:2258] T 38f5e5cac6e44cc39a4484d96f16791b P 836c226145af4934b2c23b6ea6a8ed51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:14:33.410110 14387 raft_consensus.cc:2258] T 38f5e5cac6e44cc39a4484d96f16791b P 28bad0fe80174a468e9a3ded48416aa3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:14:33.410451 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 13950
I20260805 23:14:33.413035 14387 ts_tablet_manager.cc:1905] T 38f5e5cac6e44cc39a4484d96f16791b P 28bad0fe80174a468e9a3ded48416aa3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:14:33.413450 14386 ts_tablet_manager.cc:1905] T 38f5e5cac6e44cc39a4484d96f16791b P 836c226145af4934b2c23b6ea6a8ed51: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:14:33.416085 14387 ts_tablet_manager.cc:1918] T 38f5e5cac6e44cc39a4484d96f16791b P 28bad0fe80174a468e9a3ded48416aa3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 23:14:33.416214 14387 log.cc:1196] T 38f5e5cac6e44cc39a4484d96f16791b P 28bad0fe80174a468e9a3ded48416aa3: Deleting WAL directory at /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971406251336-25971-0/minicluster-data/ts-1/wal/wals/38f5e5cac6e44cc39a4484d96f16791b
I20260805 23:14:33.416224 14386 ts_tablet_manager.cc:1918] T 38f5e5cac6e44cc39a4484d96f16791b P 836c226145af4934b2c23b6ea6a8ed51: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 23:14:33.416308 14386 log.cc:1196] T 38f5e5cac6e44cc39a4484d96f16791b P 836c226145af4934b2c23b6ea6a8ed51: Deleting WAL directory at /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971406251336-25971-0/minicluster-data/ts-2/wal/wals/38f5e5cac6e44cc39a4484d96f16791b
I20260805 23:14:33.416584 14387 ts_tablet_manager.cc:1939] T 38f5e5cac6e44cc39a4484d96f16791b P 28bad0fe80174a468e9a3ded48416aa3: Deleting consensus metadata
I20260805 23:14:33.416635 14386 ts_tablet_manager.cc:1939] T 38f5e5cac6e44cc39a4484d96f16791b P 836c226145af4934b2c23b6ea6a8ed51: Deleting consensus metadata
I20260805 23:14:33.417935 13872 catalog_manager.cc:4841] TS 836c226145af4934b2c23b6ea6a8ed51 (127.25.92.195:45193): tablet 38f5e5cac6e44cc39a4484d96f16791b (table test-table [id=f92676b1d23f4ef286d92a3806908ca7]) successfully deleted
I20260805 23:14:33.418011 13871 catalog_manager.cc:4841] TS 28bad0fe80174a468e9a3ded48416aa3 (127.25.92.194:42555): tablet 38f5e5cac6e44cc39a4484d96f16791b (table test-table [id=f92676b1d23f4ef286d92a3806908ca7]) successfully deleted
W20260805 23:14:33.420434 13874 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.25.92.193:36589 (error 108)
I20260805 23:14:33.421402 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 14082
I20260805 23:14:33.421901 14358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:33.421346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.193:36589 (local address 127.0.0.1:38930)
0805 23:14:33.421654 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0805 23:14:33.421783 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.193:36589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260805 23:14:33.426311 13874 catalog_manager.cc:4584] TS 65f54374bb744e9fb4d8ac7b78eb0dcd (127.25.92.193:36589): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 38f5e5cac6e44cc39a4484d96f16791b: Network error: Client connection negotiation failed: client connection to 127.25.92.193:36589: connect: Connection refused (error 111)
I20260805 23:14:33.431507 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 14216
I20260805 23:14:33.437741 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 13855
2026-08-05T23:14:33Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1108 ms)
[----------] 6 tests from EncryptionPolicyTest (11531 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785971406251336-25971-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 23:14:33 dist-test-slave-ss4n krb5kdc[14392](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 23:14:33 dist-test-slave-ss4n krb5kdc[14392](info): set up 2 sockets
Aug 05 23:14:33 dist-test-slave-ss4n krb5kdc[14392](info): commencing operation
krb5kdc: starting...
W20260805 23:14:34.548178 25971 mini_kdc.cc:121] Time spent starting KDC: real 1.087s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785971406251336-25971-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785971406251336-25971-0/krb5kdc/test-user.keytab.
Aug 05 23:14:34 dist-test-slave-ss4n krb5kdc[14392](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971674, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T23:14:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:14:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785971406251336-25971-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785971406251336-25971-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260805 23:14:34.686085 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785971406251336-25971-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785971406251336-25971-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785971406251336-25971-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42801
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785971406251336-25971-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42801
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:14:34.804899 14405 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:14:34.805261 14405 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:14:34.805339 14405 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:14:34.809671 14405 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:14:34.809790 14405 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:14:34.809830 14405 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:14:34.809866 14405 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:14:34.809904 14405 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:14:34.816009 14405 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785971406251336-25971-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785971406251336-25971-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42801
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785971406251336-25971-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42801
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14405
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785971406251336-25971-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:14:34.817270 14405 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:14:34.818449 14405 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:14:34.827636 14411 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:14:34.827692 14413 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:14:34.827647 14410 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:14:34.828052 14405 server_base.cc:1034] running on GCE node
I20260805 23:14:34.828667 14405 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:14:34.829977 14405 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:14:34.831216 14405 hybrid_clock.cc:648] HybridClock initialized: now 1785971674831203 us; error 28 us; skew 500 ppm
Aug 05 23:14:34 dist-test-slave-ss4n krb5kdc[14392](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971674, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:14:34.838668 14405 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260805 23:14:34.839824 14405 webserver.cc:460] Webserver started at http://127.0.0.1:39521/ using document root <none> and password file <none>
I20260805 23:14:34.840449 14405 fs_manager.cc:359] Metadata directory not provided
I20260805 23:14:34.840545 14405 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:14:34.840783 14405 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:14:34.843163 14405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785971406251336-25971-0/minicluster-data/master-0/data/instance:
uuid: "1c4846785b1e41ab85ba8bddf5d7aed2"
format_stamp: "Formatted at 2026-08-05 23:14:34 on dist-test-slave-ss4n"
server_key: "c9e2e8c8b224cdc63e7c5f9ea023bda0"
server_key_iv: "15e519aa16fd6078d85e4354209e3e36"
server_key_version: "encryptionkey@0"
I20260805 23:14:34.843799 14405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785971406251336-25971-0/minicluster-data/master-0/wal/instance:
uuid: "1c4846785b1e41ab85ba8bddf5d7aed2"
format_stamp: "Formatted at 2026-08-05 23:14:34 on dist-test-slave-ss4n"
server_key: "c9e2e8c8b224cdc63e7c5f9ea023bda0"
server_key_iv: "15e519aa16fd6078d85e4354209e3e36"
server_key_version: "encryptionkey@0"
I20260805 23:14:34.848057 14405 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260805 23:14:34.850839 14420 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:14:34.852519 14405 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260805 23:14:34.852720 14405 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785971406251336-25971-0/minicluster-data/master-0/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785971406251336-25971-0/minicluster-data/master-0/wal
uuid: "1c4846785b1e41ab85ba8bddf5d7aed2"
format_stamp: "Formatted at 2026-08-05 23:14:34 on dist-test-slave-ss4n"
server_key: "c9e2e8c8b224cdc63e7c5f9ea023bda0"
server_key_iv: "15e519aa16fd6078d85e4354209e3e36"
server_key_version: "encryptionkey@0"
I20260805 23:14:34.852885 14405 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785971406251336-25971-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785971406251336-25971-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785971406251336-25971-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:14:34.867695 14405 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:14:34.871114 14405 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:14:34.871289 14405 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:14:34.879227 14405 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42801
I20260805 23:14:34.879241 14474 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42801 every 8 connection(s)
I20260805 23:14:34.880327 14405 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
I20260805 23:14:34.883399 14475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:14:34.883697 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 14405
I20260805 23:14:34.883852 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785971406251336-25971-0/minicluster-data/master-0/wal/instance
I20260805 23:14:34.884168 25971 external_mini_cluster.cc:1428] Setting key e3c8c2e2980ee7ec145675b48a09978a
I20260805 23:14:34.890563 14475 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c4846785b1e41ab85ba8bddf5d7aed2: Bootstrap starting.
I20260805 23:14:34.893296 14475 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1c4846785b1e41ab85ba8bddf5d7aed2: Neither blocks nor log segments found. Creating new log.
Aug 05 23:14:34 dist-test-slave-ss4n krb5kdc[14392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971674, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260805 23:14:34.894320 14475 log.cc:824] T 00000000000000000000000000000000 P 1c4846785b1e41ab85ba8bddf5d7aed2: Log is configured to *not* fsync() on all Append() calls
I20260805 23:14:34.896739 14475 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c4846785b1e41ab85ba8bddf5d7aed2: No bootstrap required, opened a new log
I20260805 23:14:34.899662 14478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:34.885754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34717 (local address 127.0.0.1:42801)
0805 23:14:34.886246 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0805 23:14:34.886261 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0805 23:14:34.886296 (+ 35us) server_negotiation.cc:407] Connection header received
0805 23:14:34.887045 (+ 749us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:14:34.887064 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:14:34.887406 (+ 342us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:14:34.887756 (+ 350us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:14:34.888631 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:34.889710 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:34.890581 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:34.891103 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:34.891275 (+ 172us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:34.894386 (+ 3111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:14:34.894391 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:14:34.894406 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:14:34.894463 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:14:34.896403 (+ 1940us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:14:34.897041 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:14:34.897046 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:14:34.897051 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:14:34.897166 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:14:34.897629 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:14:34.897632 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:14:34.897633 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:14:34.898041 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:14:34.898209 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0805 23:14:34.898625 (+ 416us) server_negotiation.cc:299] Negotiation successful
0805 23:14:34.898886 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":347,"thread_start_us":175,"threads_started":1}
I20260805 23:14:34.901144 14475 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1c4846785b1e41ab85ba8bddf5d7aed2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c4846785b1e41ab85ba8bddf5d7aed2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42801 } }
I20260805 23:14:34.901424 14475 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1c4846785b1e41ab85ba8bddf5d7aed2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:14:34.901486 14475 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1c4846785b1e41ab85ba8bddf5d7aed2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c4846785b1e41ab85ba8bddf5d7aed2, State: Initialized, Role: FOLLOWER
I20260805 23:14:34.902021 14475 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1c4846785b1e41ab85ba8bddf5d7aed2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c4846785b1e41ab85ba8bddf5d7aed2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42801 } }
I20260805 23:14:34.902218 14475 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1c4846785b1e41ab85ba8bddf5d7aed2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:14:34.902324 14475 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1c4846785b1e41ab85ba8bddf5d7aed2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:14:34.902463 14475 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1c4846785b1e41ab85ba8bddf5d7aed2 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:14:34.902729 25971 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260805 23:14:34.903601 14475 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1c4846785b1e41ab85ba8bddf5d7aed2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c4846785b1e41ab85ba8bddf5d7aed2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42801 } }
I20260805 23:14:34.904062 14475 leader_election.cc:302] T 00000000000000000000000000000000 P 1c4846785b1e41ab85ba8bddf5d7aed2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1c4846785b1e41ab85ba8bddf5d7aed2; no voters:
I20260805 23:14:34.904414 14475 leader_election.cc:288] T 00000000000000000000000000000000 P 1c4846785b1e41ab85ba8bddf5d7aed2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 23:14:34.904538 14484 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1c4846785b1e41ab85ba8bddf5d7aed2 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:14:34.904793 14484 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1c4846785b1e41ab85ba8bddf5d7aed2 [term 1 LEADER]: Becoming Leader. State: Replica: 1c4846785b1e41ab85ba8bddf5d7aed2, State: Running, Role: LEADER
I20260805 23:14:34.905148 14484 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1c4846785b1e41ab85ba8bddf5d7aed2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c4846785b1e41ab85ba8bddf5d7aed2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42801 } }
I20260805 23:14:34.906038 14475 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1c4846785b1e41ab85ba8bddf5d7aed2 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:14:34.907136 14485 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c4846785b1e41ab85ba8bddf5d7aed2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1c4846785b1e41ab85ba8bddf5d7aed2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c4846785b1e41ab85ba8bddf5d7aed2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42801 } } }
I20260805 23:14:34.907265 14485 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c4846785b1e41ab85ba8bddf5d7aed2 [sys.catalog]: This master's current role is: LEADER
I20260805 23:14:34.907588 14486 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c4846785b1e41ab85ba8bddf5d7aed2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1c4846785b1e41ab85ba8bddf5d7aed2. Latest consensus state: current_term: 1 leader_uuid: "1c4846785b1e41ab85ba8bddf5d7aed2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c4846785b1e41ab85ba8bddf5d7aed2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42801 } } }
I20260805 23:14:34.907704 14486 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c4846785b1e41ab85ba8bddf5d7aed2 [sys.catalog]: This master's current role is: LEADER
I20260805 23:14:34.907970 14494 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:14:34.912128 14494 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:14:34.915529 14478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:34.905278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46101 (local address 127.0.0.1:42801)
0805 23:14:34.905545 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0805 23:14:34.905551 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:14:34.905570 (+ 19us) server_negotiation.cc:407] Connection header received
0805 23:14:34.905616 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:14:34.905619 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:14:34.905670 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:14:34.905771 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:14:34.906608 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:34.907458 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:34.908709 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:34.909204 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:34.909306 (+ 102us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:34.911283 (+ 1977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:14:34.911288 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:14:34.911295 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:14:34.911323 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:14:34.913515 (+ 2192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:14:34.914185 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:14:34.914189 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:14:34.914190 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:14:34.914280 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:14:34.914675 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:14:34.914677 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:14:34.914679 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:14:34.914854 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:14:34.914966 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0805 23:14:34.915298 (+ 332us) server_negotiation.cc:299] Negotiation successful
0805 23:14:34.915377 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260805 23:14:34.918432 14494 catalog_manager.cc:1347] Generated new cluster ID: d1295df051b9407faa22230b7e4758af
I20260805 23:14:34.918505 14494 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:14:34.937726 14494 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:14:34.939200 14494 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:14:34.946667 14494 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1c4846785b1e41ab85ba8bddf5d7aed2: Generated new TSK 0
I20260805 23:14:34.947638 14494 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260805 23:14:34.958911 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 14405
W20260805 23:14:34.965164 14390 connection.cc:535] client connection to 127.0.0.1:42801 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42801): Connection reset by peer (error 104)
2026-08-05T23:14:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1519 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785971406251336-25971-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 23:14:35 dist-test-slave-ss4n krb5kdc[14506](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 23:14:35 dist-test-slave-ss4n krb5kdc[14506](info): set up 2 sockets
Aug 05 23:14:35 dist-test-slave-ss4n krb5kdc[14506](info): commencing operation
krb5kdc: starting...
W20260805 23:14:36.056519 25971 mini_kdc.cc:121] Time spent starting KDC: real 1.077s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785971406251336-25971-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785971406251336-25971-0/krb5kdc/test-user.keytab.
Aug 05 23:14:36 dist-test-slave-ss4n krb5kdc[14506](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971676, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T23:14:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:14:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785971406251336-25971-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785971406251336-25971-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260805 23:14:36.194046 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785971406251336-25971-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785971406251336-25971-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785971406251336-25971-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35445
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785971406251336-25971-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38347
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35445
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:14:36.318140 14519 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:14:36.318550 14519 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:14:36.318662 14519 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:14:36.324410 14519 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:14:36.324589 14519 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:14:36.324635 14519 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:14:36.324699 14519 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:14:36.324739 14519 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:14:36.331187 14519 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38347
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785971406251336-25971-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785971406251336-25971-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35445
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785971406251336-25971-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35445
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14519
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785971406251336-25971-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:14:36.332427 14519 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:14:36.333540 14519 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:14:36.343246 14527 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:14:36.343246 14524 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:14:36.343417 14519 server_base.cc:1034] running on GCE node
W20260805 23:14:36.343339 14525 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:14:36.344358 14519 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:14:36.345733 14519 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:14:36.346969 14519 hybrid_clock.cc:648] HybridClock initialized: now 1785971676346918 us; error 85 us; skew 500 ppm
Aug 05 23:14:36 dist-test-slave-ss4n krb5kdc[14506](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971676, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:14:36.350735 14519 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260805 23:14:36.352643 14519 webserver.cc:460] Webserver started at http://127.0.0.1:40851/ using document root <none> and password file <none>
I20260805 23:14:36.353384 14519 fs_manager.cc:359] Metadata directory not provided
I20260805 23:14:36.353542 14519 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:14:36.353799 14519 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:14:36.356166 14519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785971406251336-25971-0/minicluster-data/master-0/data/instance:
uuid: "7246e04eaff94d2db5eb5ec1e5d775a2"
format_stamp: "Formatted at 2026-08-05 23:14:36 on dist-test-slave-ss4n"
server_key: "18654e35c45a041112acae737af89298"
server_key_iv: "e8d0f1a0203786e295bbcf6a20cd9267"
server_key_version: "encryptionkey@0"
I20260805 23:14:36.356827 14519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785971406251336-25971-0/minicluster-data/master-0/wal/instance:
uuid: "7246e04eaff94d2db5eb5ec1e5d775a2"
format_stamp: "Formatted at 2026-08-05 23:14:36 on dist-test-slave-ss4n"
server_key: "18654e35c45a041112acae737af89298"
server_key_iv: "e8d0f1a0203786e295bbcf6a20cd9267"
server_key_version: "encryptionkey@0"
I20260805 23:14:36.361198 14519 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260805 23:14:36.364135 14534 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:14:36.366289 14519 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:14:36.366510 14519 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785971406251336-25971-0/minicluster-data/master-0/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785971406251336-25971-0/minicluster-data/master-0/wal
uuid: "7246e04eaff94d2db5eb5ec1e5d775a2"
format_stamp: "Formatted at 2026-08-05 23:14:36 on dist-test-slave-ss4n"
server_key: "18654e35c45a041112acae737af89298"
server_key_iv: "e8d0f1a0203786e295bbcf6a20cd9267"
server_key_version: "encryptionkey@0"
I20260805 23:14:36.366673 14519 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785971406251336-25971-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785971406251336-25971-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785971406251336-25971-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:14:36.396224 14519 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:14:36.400179 14519 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:14:36.400434 14519 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:14:36.409821 14519 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35445
I20260805 23:14:36.410166 14586 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35445 every 8 connection(s)
I20260805 23:14:36.410996 14519 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
I20260805 23:14:36.412580 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 14519
I20260805 23:14:36.412750 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785971406251336-25971-0/minicluster-data/master-0/wal/instance
I20260805 23:14:36.413085 25971 external_mini_cluster.cc:1428] Setting key 324f641fee702e3b3886845950d2b8b2
I20260805 23:14:36.415359 14587 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:14:36.422788 14587 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7246e04eaff94d2db5eb5ec1e5d775a2: Bootstrap starting.
I20260805 23:14:36.423545 14590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:36.414697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44283 (local address 127.0.0.1:35445)
0805 23:14:36.415173 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0805 23:14:36.415190 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0805 23:14:36.415230 (+ 40us) server_negotiation.cc:407] Connection header received
0805 23:14:36.415959 (+ 729us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:14:36.415977 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:14:36.416339 (+ 362us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:14:36.416652 (+ 313us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:14:36.417216 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:36.418388 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:36.419674 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:36.420202 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:36.420352 (+ 150us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:36.421438 (+ 1086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:14:36.421443 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:14:36.421459 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:14:36.422011 (+ 552us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:14:36.422078 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:14:36.422105 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:14:36.422236 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0805 23:14:36.422452 (+ 216us) server_negotiation.cc:299] Negotiation successful
0805 23:14:36.422686 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":171,"threads_started":1}
I20260805 23:14:36.425474 14587 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7246e04eaff94d2db5eb5ec1e5d775a2: Neither blocks nor log segments found. Creating new log.
I20260805 23:14:36.426402 14587 log.cc:824] T 00000000000000000000000000000000 P 7246e04eaff94d2db5eb5ec1e5d775a2: Log is configured to *not* fsync() on all Append() calls
I20260805 23:14:36.427029 25971 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260805 23:14:36.429103 14587 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7246e04eaff94d2db5eb5ec1e5d775a2: No bootstrap required, opened a new log
I20260805 23:14:36.434012 14587 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7246e04eaff94d2db5eb5ec1e5d775a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7246e04eaff94d2db5eb5ec1e5d775a2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35445 } }
I20260805 23:14:36.434391 14587 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7246e04eaff94d2db5eb5ec1e5d775a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:14:36.434428 14590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:36.429592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56619 (local address 127.0.0.1:35445)
0805 23:14:36.429725 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0805 23:14:36.429729 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:14:36.429989 (+ 260us) server_negotiation.cc:407] Connection header received
0805 23:14:36.430140 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:14:36.430143 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:14:36.430193 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:14:36.430291 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:14:36.430592 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:36.431396 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:36.432403 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:36.432761 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:36.432889 (+ 128us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:36.433914 (+ 1025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:14:36.433918 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:14:36.433921 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:14:36.433937 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:14:36.433969 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:14:36.433983 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:14:36.434084 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0805 23:14:36.434240 (+ 156us) server_negotiation.cc:299] Negotiation successful
0805 23:14:36.434283 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260805 23:14:36.434456 14587 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7246e04eaff94d2db5eb5ec1e5d775a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7246e04eaff94d2db5eb5ec1e5d775a2, State: Initialized, Role: FOLLOWER
I20260805 23:14:36.434962 14587 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7246e04eaff94d2db5eb5ec1e5d775a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7246e04eaff94d2db5eb5ec1e5d775a2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35445 } }
I20260805 23:14:36.435138 14587 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7246e04eaff94d2db5eb5ec1e5d775a2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:14:36.435215 14587 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7246e04eaff94d2db5eb5ec1e5d775a2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:14:36.435323 14587 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7246e04eaff94d2db5eb5ec1e5d775a2 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:14:36.436357 14587 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7246e04eaff94d2db5eb5ec1e5d775a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7246e04eaff94d2db5eb5ec1e5d775a2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35445 } }
I20260805 23:14:36.436738 14587 leader_election.cc:302] T 00000000000000000000000000000000 P 7246e04eaff94d2db5eb5ec1e5d775a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7246e04eaff94d2db5eb5ec1e5d775a2; no voters:
I20260805 23:14:36.437091 14587 leader_election.cc:288] T 00000000000000000000000000000000 P 7246e04eaff94d2db5eb5ec1e5d775a2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 23:14:36.437294 14597 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7246e04eaff94d2db5eb5ec1e5d775a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:14:36.437578 14597 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7246e04eaff94d2db5eb5ec1e5d775a2 [term 1 LEADER]: Becoming Leader. State: Replica: 7246e04eaff94d2db5eb5ec1e5d775a2, State: Running, Role: LEADER
I20260805 23:14:36.437901 14597 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7246e04eaff94d2db5eb5ec1e5d775a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7246e04eaff94d2db5eb5ec1e5d775a2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35445 } }
I20260805 23:14:36.438505 14587 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7246e04eaff94d2db5eb5ec1e5d775a2 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:14:36.439769 14598 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7246e04eaff94d2db5eb5ec1e5d775a2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7246e04eaff94d2db5eb5ec1e5d775a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7246e04eaff94d2db5eb5ec1e5d775a2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35445 } } }
I20260805 23:14:36.439779 14599 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7246e04eaff94d2db5eb5ec1e5d775a2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7246e04eaff94d2db5eb5ec1e5d775a2. Latest consensus state: current_term: 1 leader_uuid: "7246e04eaff94d2db5eb5ec1e5d775a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7246e04eaff94d2db5eb5ec1e5d775a2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35445 } } }
I20260805 23:14:36.439942 14598 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7246e04eaff94d2db5eb5ec1e5d775a2 [sys.catalog]: This master's current role is: LEADER
I20260805 23:14:36.439946 14599 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7246e04eaff94d2db5eb5ec1e5d775a2 [sys.catalog]: This master's current role is: LEADER
I20260805 23:14:36.440357 14606 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:14:36.443786 14606 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:14:36.450460 14606 catalog_manager.cc:1347] Generated new cluster ID: f0dc5f896e504a97aacd5e5ec41a17ee
I20260805 23:14:36.450584 14606 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:14:36.465070 14606 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:14:36.466279 14606 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:14:36.470775 14606 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7246e04eaff94d2db5eb5ec1e5d775a2: Generated new TSK 0
I20260805 23:14:36.471700 14606 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260805 23:14:36.486732 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 14519
W20260805 23:14:36.492909 14504 connection.cc:535] client connection to 127.0.0.1:35445 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35445): Connection reset by peer (error 104)
2026-08-05T23:14:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1528 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785971406251336-25971-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 23:14:36 dist-test-slave-ss4n krb5kdc[14619](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 23:14:36 dist-test-slave-ss4n krb5kdc[14619](info): set up 2 sockets
Aug 05 23:14:36 dist-test-slave-ss4n krb5kdc[14619](info): commencing operation
krb5kdc: starting...
W20260805 23:14:37.575510 25971 mini_kdc.cc:121] Time spent starting KDC: real 1.067s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785971406251336-25971-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785971406251336-25971-0/krb5kdc/test-user.keytab.
Aug 05 23:14:37 dist-test-slave-ss4n krb5kdc[14619](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971677, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T23:14:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:14:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785971406251336-25971-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785971406251336-25971-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260805 23:14:37.714639 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785971406251336-25971-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785971406251336-25971-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785971406251336-25971-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44499
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785971406251336-25971-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44499
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:14:37.838816 14632 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:14:37.839195 14632 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:14:37.839286 14632 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:14:37.843604 14632 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:14:37.843761 14632 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:14:37.843801 14632 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:14:37.843851 14632 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:14:37.843890 14632 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:14:37.849691 14632 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785971406251336-25971-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785971406251336-25971-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44499
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785971406251336-25971-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44499
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14632
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785971406251336-25971-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:14:37.850970 14632 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:14:37.852080 14632 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:14:37.861586 14638 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:14:37.861589 14640 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:14:37.861591 14637 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:14:37.861882 14632 server_base.cc:1034] running on GCE node
I20260805 23:14:37.862530 14632 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:14:37.863783 14632 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:14:37.864998 14632 hybrid_clock.cc:648] HybridClock initialized: now 1785971677864962 us; error 49 us; skew 500 ppm
Aug 05 23:14:37 dist-test-slave-ss4n krb5kdc[14619](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971677, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:14:37.868499 14632 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260805 23:14:37.869596 14632 webserver.cc:460] Webserver started at http://127.0.0.1:39411/ using document root <none> and password file <none>
I20260805 23:14:37.870237 14632 fs_manager.cc:359] Metadata directory not provided
I20260805 23:14:37.870342 14632 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:14:37.870610 14632 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:14:37.872850 14632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785971406251336-25971-0/minicluster-data/master-0/data/instance:
uuid: "9b8b826d81354f7094d12c0b50f25c29"
format_stamp: "Formatted at 2026-08-05 23:14:37 on dist-test-slave-ss4n"
server_key: "b1ab5060eb38a9f2cde5b4723b104ccf"
server_key_iv: "d993c4c7a730789019b216779a506c88"
server_key_version: "encryptionkey@0"
I20260805 23:14:37.873454 14632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785971406251336-25971-0/minicluster-data/master-0/wal/instance:
uuid: "9b8b826d81354f7094d12c0b50f25c29"
format_stamp: "Formatted at 2026-08-05 23:14:37 on dist-test-slave-ss4n"
server_key: "b1ab5060eb38a9f2cde5b4723b104ccf"
server_key_iv: "d993c4c7a730789019b216779a506c88"
server_key_version: "encryptionkey@0"
I20260805 23:14:37.878024 14632 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260805 23:14:37.880633 14647 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:14:37.882272 14632 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:14:37.882448 14632 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785971406251336-25971-0/minicluster-data/master-0/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785971406251336-25971-0/minicluster-data/master-0/wal
uuid: "9b8b826d81354f7094d12c0b50f25c29"
format_stamp: "Formatted at 2026-08-05 23:14:37 on dist-test-slave-ss4n"
server_key: "b1ab5060eb38a9f2cde5b4723b104ccf"
server_key_iv: "d993c4c7a730789019b216779a506c88"
server_key_version: "encryptionkey@0"
I20260805 23:14:37.882599 14632 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785971406251336-25971-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785971406251336-25971-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785971406251336-25971-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:14:37.888269 14632 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:14:37.892352 14632 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:14:37.892581 14632 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:14:37.901388 14632 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44499
I20260805 23:14:37.901468 14701 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44499 every 8 connection(s)
I20260805 23:14:37.902642 14632 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
I20260805 23:14:37.905833 14702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:14:37.911742 14702 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b8b826d81354f7094d12c0b50f25c29: Bootstrap starting.
I20260805 23:14:37.912667 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 14632
I20260805 23:14:37.912891 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785971406251336-25971-0/minicluster-data/master-0/wal/instance
I20260805 23:14:37.913211 25971 external_mini_cluster.cc:1428] Setting key 9b817a4ac11283d8e7cf9e58113a66e5
I20260805 23:14:37.914964 14702 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9b8b826d81354f7094d12c0b50f25c29: Neither blocks nor log segments found. Creating new log.
I20260805 23:14:37.915799 14702 log.cc:824] T 00000000000000000000000000000000 P 9b8b826d81354f7094d12c0b50f25c29: Log is configured to *not* fsync() on all Append() calls
I20260805 23:14:37.919112 14702 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b8b826d81354f7094d12c0b50f25c29: No bootstrap required, opened a new log
I20260805 23:14:37.920585 14705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:37.914667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53371 (local address 127.0.0.1:44499)
0805 23:14:37.915279 (+ 612us) server_negotiation.cc:206] Beginning negotiation
0805 23:14:37.915303 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0805 23:14:37.915357 (+ 54us) server_negotiation.cc:407] Connection header received
0805 23:14:37.916669 (+ 1312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:14:37.916702 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:14:37.917156 (+ 454us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:14:37.917709 (+ 553us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:14:37.918743 (+ 1034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:14:37.918750 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:14:37.918768 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:14:37.918784 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:14:37.918869 (+ 85us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:14:37.918907 (+ 38us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:14:37.919094 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0805 23:14:37.919224 (+ 130us) server_negotiation.cc:299] Negotiation successful
0805 23:14:37.919440 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":149,"threads_started":1}
I20260805 23:14:37.924538 14702 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9b8b826d81354f7094d12c0b50f25c29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b8b826d81354f7094d12c0b50f25c29" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44499 } }
I20260805 23:14:37.924880 14702 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9b8b826d81354f7094d12c0b50f25c29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:14:37.924947 14702 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9b8b826d81354f7094d12c0b50f25c29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b8b826d81354f7094d12c0b50f25c29, State: Initialized, Role: FOLLOWER
I20260805 23:14:37.925412 14702 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9b8b826d81354f7094d12c0b50f25c29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b8b826d81354f7094d12c0b50f25c29" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44499 } }
I20260805 23:14:37.925637 14702 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9b8b826d81354f7094d12c0b50f25c29 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:14:37.925711 14702 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9b8b826d81354f7094d12c0b50f25c29 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:14:37.925817 14702 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9b8b826d81354f7094d12c0b50f25c29 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:14:37.926164 25971 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260805 23:14:37.927212 14702 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9b8b826d81354f7094d12c0b50f25c29 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b8b826d81354f7094d12c0b50f25c29" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44499 } }
I20260805 23:14:37.927656 14702 leader_election.cc:302] T 00000000000000000000000000000000 P 9b8b826d81354f7094d12c0b50f25c29 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9b8b826d81354f7094d12c0b50f25c29; no voters:
I20260805 23:14:37.928076 14702 leader_election.cc:288] T 00000000000000000000000000000000 P 9b8b826d81354f7094d12c0b50f25c29 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 23:14:37.928259 14711 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9b8b826d81354f7094d12c0b50f25c29 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:14:37.928568 14711 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9b8b826d81354f7094d12c0b50f25c29 [term 1 LEADER]: Becoming Leader. State: Replica: 9b8b826d81354f7094d12c0b50f25c29, State: Running, Role: LEADER
I20260805 23:14:37.928925 14711 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9b8b826d81354f7094d12c0b50f25c29 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b8b826d81354f7094d12c0b50f25c29" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44499 } }
I20260805 23:14:37.929589 14702 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9b8b826d81354f7094d12c0b50f25c29 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:14:37.930920 14713 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b8b826d81354f7094d12c0b50f25c29 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9b8b826d81354f7094d12c0b50f25c29. Latest consensus state: current_term: 1 leader_uuid: "9b8b826d81354f7094d12c0b50f25c29" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b8b826d81354f7094d12c0b50f25c29" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44499 } } }
I20260805 23:14:37.930969 14712 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b8b826d81354f7094d12c0b50f25c29 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9b8b826d81354f7094d12c0b50f25c29" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b8b826d81354f7094d12c0b50f25c29" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44499 } } }
I20260805 23:14:37.931164 14712 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b8b826d81354f7094d12c0b50f25c29 [sys.catalog]: This master's current role is: LEADER
I20260805 23:14:37.931205 14713 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b8b826d81354f7094d12c0b50f25c29 [sys.catalog]: This master's current role is: LEADER
I20260805 23:14:37.931290 14705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:37.929069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42505 (local address 127.0.0.1:44499)
0805 23:14:37.929192 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0805 23:14:37.929197 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:14:37.929213 (+ 16us) server_negotiation.cc:407] Connection header received
0805 23:14:37.929403 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:14:37.929407 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:14:37.929463 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:14:37.929562 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:14:37.930530 (+ 968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:14:37.930534 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:14:37.930537 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:14:37.930549 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:14:37.930580 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:14:37.930593 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:14:37.930682 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0805 23:14:37.931068 (+ 386us) server_negotiation.cc:299] Negotiation successful
0805 23:14:37.931113 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260805 23:14:37.931624 14721 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:14:37.935675 14721 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:14:37.941815 14721 catalog_manager.cc:1347] Generated new cluster ID: 2d03579812b541858b157a6f5c168ed6
I20260805 23:14:37.941927 14721 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:14:37.969398 14721 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:14:37.969825 14721 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:14:37.990612 14721 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9b8b826d81354f7094d12c0b50f25c29: Generated new TSK 0
I20260805 23:14:37.991626 14721 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260805 23:14:38.000147 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 14632
2026-08-05T23:14:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1513 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785971406251336-25971-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 23:14:38 dist-test-slave-ss4n krb5kdc[14733](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 23:14:38 dist-test-slave-ss4n krb5kdc[14733](info): set up 2 sockets
Aug 05 23:14:38 dist-test-slave-ss4n krb5kdc[14733](info): commencing operation
krb5kdc: starting...
W20260805 23:14:39.130961 25971 mini_kdc.cc:121] Time spent starting KDC: real 1.109s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785971406251336-25971-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785971406251336-25971-0/krb5kdc/test-user.keytab.
Aug 05 23:14:39 dist-test-slave-ss4n krb5kdc[14733](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971679, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T23:14:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:14:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785971406251336-25971-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785971406251336-25971-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260805 23:14:39.264899 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785971406251336-25971-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785971406251336-25971-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785971406251336-25971-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44049
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785971406251336-25971-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44049
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:14:39.380103 14746 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:14:39.380430 14746 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:14:39.380474 14746 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:14:39.384763 14746 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:14:39.384846 14746 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:14:39.384866 14746 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:14:39.384883 14746 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:14:39.384899 14746 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:14:39.390543 14746 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785971406251336-25971-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785971406251336-25971-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44049
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785971406251336-25971-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44049
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14746
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785971406251336-25971-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:14:39.391708 14746 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:14:39.392765 14746 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:14:39.401547 14752 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:14:39.401551 14751 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:14:39.401551 14755 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:14:39.402228 14746 server_base.cc:1034] running on GCE node
I20260805 23:14:39.402814 14746 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:14:39.403998 14746 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:14:39.405174 14746 hybrid_clock.cc:648] HybridClock initialized: now 1785971679405161 us; error 26 us; skew 500 ppm
Aug 05 23:14:39 dist-test-slave-ss4n krb5kdc[14733](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971679, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:14:39.408072 14746 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260805 23:14:39.409013 14746 webserver.cc:460] Webserver started at http://127.0.0.1:41319/ using document root <none> and password file <none>
I20260805 23:14:39.409705 14746 fs_manager.cc:359] Metadata directory not provided
I20260805 23:14:39.409801 14746 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:14:39.410048 14746 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:14:39.412209 14746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785971406251336-25971-0/minicluster-data/master-0/data/instance:
uuid: "a2e15657c40a4100a1624665fe157b66"
format_stamp: "Formatted at 2026-08-05 23:14:39 on dist-test-slave-ss4n"
server_key: "330a7ea0d6182d3be30be9743807cda5"
server_key_iv: "63d696d7a823b787df6a1db5d2c52307"
server_key_version: "encryptionkey@0"
I20260805 23:14:39.412815 14746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785971406251336-25971-0/minicluster-data/master-0/wal/instance:
uuid: "a2e15657c40a4100a1624665fe157b66"
format_stamp: "Formatted at 2026-08-05 23:14:39 on dist-test-slave-ss4n"
server_key: "330a7ea0d6182d3be30be9743807cda5"
server_key_iv: "63d696d7a823b787df6a1db5d2c52307"
server_key_version: "encryptionkey@0"
I20260805 23:14:39.416755 14746 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260805 23:14:39.419407 14762 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:14:39.420660 14746 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260805 23:14:39.420814 14746 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785971406251336-25971-0/minicluster-data/master-0/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785971406251336-25971-0/minicluster-data/master-0/wal
uuid: "a2e15657c40a4100a1624665fe157b66"
format_stamp: "Formatted at 2026-08-05 23:14:39 on dist-test-slave-ss4n"
server_key: "330a7ea0d6182d3be30be9743807cda5"
server_key_iv: "63d696d7a823b787df6a1db5d2c52307"
server_key_version: "encryptionkey@0"
I20260805 23:14:39.420954 14746 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785971406251336-25971-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785971406251336-25971-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785971406251336-25971-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:14:39.434489 14746 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:14:39.437906 14746 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:14:39.438081 14746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:14:39.445957 14746 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44049
I20260805 23:14:39.445994 14815 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44049 every 8 connection(s)
I20260805 23:14:39.447014 14746 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
I20260805 23:14:39.450186 14816 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:14:39.452358 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 14746
I20260805 23:14:39.452527 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785971406251336-25971-0/minicluster-data/master-0/wal/instance
I20260805 23:14:39.452852 25971 external_mini_cluster.cc:1428] Setting key 1920548afc320711c921c35e122de78f
I20260805 23:14:39.458350 14816 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2e15657c40a4100a1624665fe157b66: Bootstrap starting.
I20260805 23:14:39.461400 14816 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a2e15657c40a4100a1624665fe157b66: Neither blocks nor log segments found. Creating new log.
I20260805 23:14:39.462428 14816 log.cc:824] T 00000000000000000000000000000000 P a2e15657c40a4100a1624665fe157b66: Log is configured to *not* fsync() on all Append() calls
Aug 05 23:14:39 dist-test-slave-ss4n krb5kdc[14733](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971679, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260805 23:14:39.465174 14816 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2e15657c40a4100a1624665fe157b66: No bootstrap required, opened a new log
I20260805 23:14:39.470901 14816 raft_consensus.cc:348] T 00000000000000000000000000000000 P a2e15657c40a4100a1624665fe157b66 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2e15657c40a4100a1624665fe157b66" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44049 } }
I20260805 23:14:39.471233 14816 raft_consensus.cc:374] T 00000000000000000000000000000000 P a2e15657c40a4100a1624665fe157b66 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:14:39.471287 14816 raft_consensus.cc:729] T 00000000000000000000000000000000 P a2e15657c40a4100a1624665fe157b66 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2e15657c40a4100a1624665fe157b66, State: Initialized, Role: FOLLOWER
I20260805 23:14:39.471549 14819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:39.454523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35013 (local address 127.0.0.1:44049)
0805 23:14:39.455013 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0805 23:14:39.455029 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0805 23:14:39.455067 (+ 38us) server_negotiation.cc:407] Connection header received
0805 23:14:39.455739 (+ 672us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:14:39.455756 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:14:39.456112 (+ 356us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:14:39.456420 (+ 308us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:14:39.457555 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:39.459167 (+ 1612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:39.460239 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:39.460973 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:39.461184 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:39.464747 (+ 3563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:14:39.464758 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:14:39.464778 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:14:39.464814 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:14:39.467303 (+ 2489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:14:39.467909 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:14:39.467919 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:14:39.467928 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:14:39.468134 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:14:39.468725 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:14:39.468728 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:14:39.468731 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:14:39.469347 (+ 616us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:14:39.469657 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0805 23:14:39.470035 (+ 378us) server_negotiation.cc:299] Negotiation successful
0805 23:14:39.470411 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":177,"threads_started":1}
I20260805 23:14:39.471807 14816 consensus_queue.cc:260] T 00000000000000000000000000000000 P a2e15657c40a4100a1624665fe157b66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2e15657c40a4100a1624665fe157b66" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44049 } }
I20260805 23:14:39.472018 14816 raft_consensus.cc:388] T 00000000000000000000000000000000 P a2e15657c40a4100a1624665fe157b66 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:14:39.472123 14816 raft_consensus.cc:482] T 00000000000000000000000000000000 P a2e15657c40a4100a1624665fe157b66 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:14:39.472237 14816 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a2e15657c40a4100a1624665fe157b66 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:14:39.473380 14816 raft_consensus.cc:504] T 00000000000000000000000000000000 P a2e15657c40a4100a1624665fe157b66 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2e15657c40a4100a1624665fe157b66" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44049 } }
I20260805 23:14:39.473839 14816 leader_election.cc:302] T 00000000000000000000000000000000 P a2e15657c40a4100a1624665fe157b66 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a2e15657c40a4100a1624665fe157b66; no voters:
I20260805 23:14:39.474222 14816 leader_election.cc:288] T 00000000000000000000000000000000 P a2e15657c40a4100a1624665fe157b66 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 23:14:39.474375 14821 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a2e15657c40a4100a1624665fe157b66 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:14:39.474689 14821 raft_consensus.cc:686] T 00000000000000000000000000000000 P a2e15657c40a4100a1624665fe157b66 [term 1 LEADER]: Becoming Leader. State: Replica: a2e15657c40a4100a1624665fe157b66, State: Running, Role: LEADER
I20260805 23:14:39.475037 25971 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260805 23:14:39.475131 14821 consensus_queue.cc:237] T 00000000000000000000000000000000 P a2e15657c40a4100a1624665fe157b66 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2e15657c40a4100a1624665fe157b66" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44049 } }
I20260805 23:14:39.475780 14816 sys_catalog.cc:564] T 00000000000000000000000000000000 P a2e15657c40a4100a1624665fe157b66 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:14:39.476890 14823 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2e15657c40a4100a1624665fe157b66 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a2e15657c40a4100a1624665fe157b66. Latest consensus state: current_term: 1 leader_uuid: "a2e15657c40a4100a1624665fe157b66" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2e15657c40a4100a1624665fe157b66" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44049 } } }
I20260805 23:14:39.477035 14823 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2e15657c40a4100a1624665fe157b66 [sys.catalog]: This master's current role is: LEADER
I20260805 23:14:39.477444 14822 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2e15657c40a4100a1624665fe157b66 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a2e15657c40a4100a1624665fe157b66" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2e15657c40a4100a1624665fe157b66" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44049 } } }
I20260805 23:14:39.477592 14822 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2e15657c40a4100a1624665fe157b66 [sys.catalog]: This master's current role is: LEADER
I20260805 23:14:39.477929 14834 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:14:39.481575 14834 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:14:39.488075 14834 catalog_manager.cc:1347] Generated new cluster ID: e0f1a686f00e486fb09bbc33dc7b5940
I20260805 23:14:39.488240 14834 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:14:39.489378 14819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:39.478675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39925 (local address 127.0.0.1:44049)
0805 23:14:39.478974 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0805 23:14:39.478981 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0805 23:14:39.479379 (+ 398us) server_negotiation.cc:407] Connection header received
0805 23:14:39.479514 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:14:39.479517 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:14:39.479575 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:14:39.479673 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:14:39.480752 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:39.481517 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:39.482841 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:39.483202 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:39.483326 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:39.485265 (+ 1939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:14:39.485270 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:14:39.485272 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:14:39.485304 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:14:39.487353 (+ 2049us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:14:39.487842 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:14:39.487847 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:14:39.487849 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:14:39.487976 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:14:39.488335 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:14:39.488341 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:14:39.488343 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:14:39.488611 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:14:39.488760 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0805 23:14:39.489065 (+ 305us) server_negotiation.cc:299] Negotiation successful
0805 23:14:39.489207 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":209}
I20260805 23:14:39.535543 14834 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:14:39.536737 14834 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:14:39.546875 14834 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a2e15657c40a4100a1624665fe157b66: Generated new TSK 0
I20260805 23:14:39.547874 14834 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260805 23:14:39.563426 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 14746
2026-08-05T23:14:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1562 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785971406251336-25971-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 23:14:39 dist-test-slave-ss4n krb5kdc[14848](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 23:14:39 dist-test-slave-ss4n krb5kdc[14848](info): set up 2 sockets
Aug 05 23:14:39 dist-test-slave-ss4n krb5kdc[14848](info): commencing operation
krb5kdc: starting...
W20260805 23:14:40.666023 25971 mini_kdc.cc:121] Time spent starting KDC: real 1.082s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785971406251336-25971-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785971406251336-25971-0/krb5kdc/test-user.keytab.
Aug 05 23:14:40 dist-test-slave-ss4n krb5kdc[14848](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971680, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T23:14:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:14:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785971406251336-25971-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785971406251336-25971-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260805 23:14:40.801666 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785971406251336-25971-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785971406251336-25971-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785971406251336-25971-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41215
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785971406251336-25971-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41215
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:14:40.917254 14861 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:14:40.917657 14861 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:14:40.917707 14861 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:14:40.921840 14861 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:14:40.921918 14861 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:14:40.921938 14861 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:14:40.921957 14861 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:14:40.921972 14861 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:14:40.927714 14861 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785971406251336-25971-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785971406251336-25971-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41215
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785971406251336-25971-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41215
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14861
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785971406251336-25971-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:14:40.928877 14861 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:14:40.930022 14861 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:14:40.938928 14869 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:14:40.938979 14866 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:14:40.938937 14867 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:14:40.939399 14861 server_base.cc:1034] running on GCE node
I20260805 23:14:40.939976 14861 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:14:40.941150 14861 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:14:40.942348 14861 hybrid_clock.cc:648] HybridClock initialized: now 1785971680942316 us; error 47 us; skew 500 ppm
Aug 05 23:14:40 dist-test-slave-ss4n krb5kdc[14848](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971680, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:14:40.950850 14861 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260805 23:14:40.952108 14861 webserver.cc:460] Webserver started at http://127.0.0.1:44455/ using document root <none> and password file <none>
I20260805 23:14:40.952787 14861 fs_manager.cc:359] Metadata directory not provided
I20260805 23:14:40.952895 14861 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:14:40.953150 14861 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:14:40.955437 14861 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785971406251336-25971-0/minicluster-data/master-0/data/instance:
uuid: "7905638ff24943b3b96b6067fc00658b"
format_stamp: "Formatted at 2026-08-05 23:14:40 on dist-test-slave-ss4n"
server_key: "15283e6a6d5efc77012be35e12cba7dd"
server_key_iv: "739e3085c261a7bb57135d207d6dff16"
server_key_version: "encryptionkey@0"
I20260805 23:14:40.956063 14861 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785971406251336-25971-0/minicluster-data/master-0/wal/instance:
uuid: "7905638ff24943b3b96b6067fc00658b"
format_stamp: "Formatted at 2026-08-05 23:14:40 on dist-test-slave-ss4n"
server_key: "15283e6a6d5efc77012be35e12cba7dd"
server_key_iv: "739e3085c261a7bb57135d207d6dff16"
server_key_version: "encryptionkey@0"
I20260805 23:14:40.960294 14861 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260805 23:14:40.963263 14876 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:14:40.964598 14861 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260805 23:14:40.964795 14861 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785971406251336-25971-0/minicluster-data/master-0/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785971406251336-25971-0/minicluster-data/master-0/wal
uuid: "7905638ff24943b3b96b6067fc00658b"
format_stamp: "Formatted at 2026-08-05 23:14:40 on dist-test-slave-ss4n"
server_key: "15283e6a6d5efc77012be35e12cba7dd"
server_key_iv: "739e3085c261a7bb57135d207d6dff16"
server_key_version: "encryptionkey@0"
I20260805 23:14:40.964939 14861 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785971406251336-25971-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785971406251336-25971-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785971406251336-25971-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:14:40.985987 14861 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:14:40.989104 14861 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:14:40.989329 14861 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:14:40.997391 14861 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41215
I20260805 23:14:40.997411 14928 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41215 every 8 connection(s)
I20260805 23:14:40.998440 14861 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
I20260805 23:14:40.999065 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 14861
I20260805 23:14:40.999228 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785971406251336-25971-0/minicluster-data/master-0/wal/instance
I20260805 23:14:40.999513 25971 external_mini_cluster.cc:1428] Setting key 3f0214404774d65d2b01c97438e18df7
I20260805 23:14:41.001720 14929 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:14:41.008828 14929 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7905638ff24943b3b96b6067fc00658b: Bootstrap starting.
I20260805 23:14:41.010236 14932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:41.001157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35159 (local address 127.0.0.1:41215)
0805 23:14:41.001659 (+ 502us) server_negotiation.cc:206] Beginning negotiation
0805 23:14:41.001679 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0805 23:14:41.001725 (+ 46us) server_negotiation.cc:407] Connection header received
0805 23:14:41.002599 (+ 874us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:14:41.002635 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:14:41.003062 (+ 427us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:14:41.003441 (+ 379us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:14:41.003862 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:41.005355 (+ 1493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:41.006454 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:41.007160 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:41.007320 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:41.008244 (+ 924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:14:41.008250 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:14:41.008267 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:14:41.008779 (+ 512us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:14:41.008852 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:14:41.008886 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:14:41.008986 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0805 23:14:41.009200 (+ 214us) server_negotiation.cc:299] Negotiation successful
0805 23:14:41.009422 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":132,"threads_started":1}
I20260805 23:14:41.011381 14929 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7905638ff24943b3b96b6067fc00658b: Neither blocks nor log segments found. Creating new log.
I20260805 23:14:41.012269 14929 log.cc:824] T 00000000000000000000000000000000 P 7905638ff24943b3b96b6067fc00658b: Log is configured to *not* fsync() on all Append() calls
I20260805 23:14:41.013366 25971 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260805 23:14:41.014353 14929 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7905638ff24943b3b96b6067fc00658b: No bootstrap required, opened a new log
I20260805 23:14:41.017741 14929 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7905638ff24943b3b96b6067fc00658b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7905638ff24943b3b96b6067fc00658b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41215 } }
I20260805 23:14:41.017973 14929 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7905638ff24943b3b96b6067fc00658b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:14:41.018009 14929 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7905638ff24943b3b96b6067fc00658b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7905638ff24943b3b96b6067fc00658b, State: Initialized, Role: FOLLOWER
I20260805 23:14:41.018500 14929 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7905638ff24943b3b96b6067fc00658b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7905638ff24943b3b96b6067fc00658b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41215 } }
I20260805 23:14:41.018656 14929 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7905638ff24943b3b96b6067fc00658b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:14:41.018756 14929 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7905638ff24943b3b96b6067fc00658b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:14:41.018865 14929 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7905638ff24943b3b96b6067fc00658b [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:14:41.019964 14932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:41.015686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42319 (local address 127.0.0.1:41215)
0805 23:14:41.015804 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0805 23:14:41.015808 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:14:41.015939 (+ 131us) server_negotiation.cc:407] Connection header received
0805 23:14:41.016103 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:14:41.016106 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:14:41.016156 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:14:41.016242 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:14:41.016521 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:41.017393 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:41.018377 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:41.018624 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:41.018744 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:41.019556 (+ 812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:14:41.019559 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:14:41.019561 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:14:41.019575 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:14:41.019600 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:14:41.019610 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:14:41.019663 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0805 23:14:41.019776 (+ 113us) server_negotiation.cc:299] Negotiation successful
0805 23:14:41.019825 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260805 23:14:41.020248 14929 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7905638ff24943b3b96b6067fc00658b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7905638ff24943b3b96b6067fc00658b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41215 } }
I20260805 23:14:41.020609 14929 leader_election.cc:302] T 00000000000000000000000000000000 P 7905638ff24943b3b96b6067fc00658b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7905638ff24943b3b96b6067fc00658b; no voters:
I20260805 23:14:41.020969 14929 leader_election.cc:288] T 00000000000000000000000000000000 P 7905638ff24943b3b96b6067fc00658b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 23:14:41.021068 14939 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7905638ff24943b3b96b6067fc00658b [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:14:41.021251 14939 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7905638ff24943b3b96b6067fc00658b [term 1 LEADER]: Becoming Leader. State: Replica: 7905638ff24943b3b96b6067fc00658b, State: Running, Role: LEADER
I20260805 23:14:41.021643 14939 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7905638ff24943b3b96b6067fc00658b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7905638ff24943b3b96b6067fc00658b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41215 } }
I20260805 23:14:41.022354 14929 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7905638ff24943b3b96b6067fc00658b [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:14:41.023411 14940 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7905638ff24943b3b96b6067fc00658b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7905638ff24943b3b96b6067fc00658b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7905638ff24943b3b96b6067fc00658b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41215 } } }
I20260805 23:14:41.023555 14940 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7905638ff24943b3b96b6067fc00658b [sys.catalog]: This master's current role is: LEADER
I20260805 23:14:41.023828 14941 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7905638ff24943b3b96b6067fc00658b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7905638ff24943b3b96b6067fc00658b. Latest consensus state: current_term: 1 leader_uuid: "7905638ff24943b3b96b6067fc00658b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7905638ff24943b3b96b6067fc00658b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41215 } } }
I20260805 23:14:41.023926 14941 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7905638ff24943b3b96b6067fc00658b [sys.catalog]: This master's current role is: LEADER
I20260805 23:14:41.023929 14948 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:14:41.026891 14948 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:14:41.032702 14948 catalog_manager.cc:1347] Generated new cluster ID: 14328582653742ef99dadd35c662b285
I20260805 23:14:41.032789 14948 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:14:41.055104 14948 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:14:41.056110 14948 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:14:41.064244 14948 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7905638ff24943b3b96b6067fc00658b: Generated new TSK 0
I20260805 23:14:41.064988 14948 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260805 23:14:41.079066 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 14861
2026-08-05T23:14:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1515 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785971406251336-25971-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 23:14:41 dist-test-slave-ss4n krb5kdc[14960](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 23:14:41 dist-test-slave-ss4n krb5kdc[14960](info): set up 2 sockets
Aug 05 23:14:41 dist-test-slave-ss4n krb5kdc[14960](info): commencing operation
krb5kdc: starting...
W20260805 23:14:42.141264 25971 mini_kdc.cc:121] Time spent starting KDC: real 1.041s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785971406251336-25971-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785971406251336-25971-0/krb5kdc/test-user.keytab.
Aug 05 23:14:42 dist-test-slave-ss4n krb5kdc[14960](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971682, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T23:14:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:14:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785971406251336-25971-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785971406251336-25971-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260805 23:14:42.278087 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785971406251336-25971-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785971406251336-25971-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785971406251336-25971-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33219
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785971406251336-25971-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38673
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33219
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:14:42.397600 14973 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:14:42.397931 14973 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:14:42.397976 14973 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:14:42.402329 14973 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:14:42.402415 14973 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:14:42.402434 14973 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:14:42.402451 14973 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:14:42.402467 14973 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:14:42.408499 14973 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38673
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785971406251336-25971-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785971406251336-25971-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33219
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785971406251336-25971-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33219
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14973
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785971406251336-25971-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:14:42.409834 14973 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:14:42.410967 14973 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260805 23:14:42.420373 14973 server_base.cc:1034] running on GCE node
W20260805 23:14:42.420313 14978 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:14:42.420313 14981 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:14:42.420318 14979 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:14:42.421139 14973 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:14:42.422501 14973 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:14:42.423713 14973 hybrid_clock.cc:648] HybridClock initialized: now 1785971682423639 us; error 87 us; skew 500 ppm
Aug 05 23:14:42 dist-test-slave-ss4n krb5kdc[14960](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971682, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:14:42.427553 14973 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260805 23:14:42.428771 14973 webserver.cc:460] Webserver started at http://127.0.0.1:42299/ using document root <none> and password file <none>
I20260805 23:14:42.429441 14973 fs_manager.cc:359] Metadata directory not provided
I20260805 23:14:42.429610 14973 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:14:42.429903 14973 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:14:42.432371 14973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785971406251336-25971-0/minicluster-data/master-0/data/instance:
uuid: "6683e0f439444cd19dbb6e739c8d35e6"
format_stamp: "Formatted at 2026-08-05 23:14:42 on dist-test-slave-ss4n"
server_key: "114dc53ebfa6d2eccf954b5f787b82ef"
server_key_iv: "4f74c0a00edcda447863c1f52c4c6893"
server_key_version: "encryptionkey@0"
I20260805 23:14:42.433089 14973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785971406251336-25971-0/minicluster-data/master-0/wal/instance:
uuid: "6683e0f439444cd19dbb6e739c8d35e6"
format_stamp: "Formatted at 2026-08-05 23:14:42 on dist-test-slave-ss4n"
server_key: "114dc53ebfa6d2eccf954b5f787b82ef"
server_key_iv: "4f74c0a00edcda447863c1f52c4c6893"
server_key_version: "encryptionkey@0"
I20260805 23:14:42.437825 14973 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.001s
I20260805 23:14:42.440685 14989 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:14:42.442289 14973 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:14:42.442451 14973 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785971406251336-25971-0/minicluster-data/master-0/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785971406251336-25971-0/minicluster-data/master-0/wal
uuid: "6683e0f439444cd19dbb6e739c8d35e6"
format_stamp: "Formatted at 2026-08-05 23:14:42 on dist-test-slave-ss4n"
server_key: "114dc53ebfa6d2eccf954b5f787b82ef"
server_key_iv: "4f74c0a00edcda447863c1f52c4c6893"
server_key_version: "encryptionkey@0"
I20260805 23:14:42.442595 14973 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785971406251336-25971-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785971406251336-25971-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785971406251336-25971-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:14:42.447862 14973 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:14:42.451943 14973 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:14:42.452203 14973 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:14:42.460536 14973 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33219
I20260805 23:14:42.460565 15043 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33219 every 8 connection(s)
I20260805 23:14:42.461678 14973 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
I20260805 23:14:42.464577 15044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:14:42.465898 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 14973
I20260805 23:14:42.466066 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785971406251336-25971-0/minicluster-data/master-0/wal/instance
I20260805 23:14:42.466411 25971 external_mini_cluster.cc:1428] Setting key 3b67ef14958cf8c6e5bf61755251a8c5
I20260805 23:14:42.471405 15044 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6683e0f439444cd19dbb6e739c8d35e6: Bootstrap starting.
I20260805 23:14:42.472335 15047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:42.468023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57343 (local address 127.0.0.1:33219)
0805 23:14:42.468522 (+ 499us) server_negotiation.cc:206] Beginning negotiation
0805 23:14:42.468540 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0805 23:14:42.468587 (+ 47us) server_negotiation.cc:407] Connection header received
0805 23:14:42.469281 (+ 694us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:14:42.469298 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:14:42.469680 (+ 382us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:14:42.470004 (+ 324us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:14:42.470953 (+ 949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:14:42.470958 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:14:42.470970 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:14:42.470981 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:14:42.471042 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:14:42.471062 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:14:42.471188 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0805 23:14:42.471310 (+ 122us) server_negotiation.cc:299] Negotiation successful
0805 23:14:42.471503 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":331,"thread_start_us":163,"threads_started":1}
I20260805 23:14:42.473954 15044 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6683e0f439444cd19dbb6e739c8d35e6: Neither blocks nor log segments found. Creating new log.
I20260805 23:14:42.474840 15044 log.cc:824] T 00000000000000000000000000000000 P 6683e0f439444cd19dbb6e739c8d35e6: Log is configured to *not* fsync() on all Append() calls
I20260805 23:14:42.475473 25971 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260805 23:14:42.477013 15044 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6683e0f439444cd19dbb6e739c8d35e6: No bootstrap required, opened a new log
I20260805 23:14:42.479489 15047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:42.477837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57809 (local address 127.0.0.1:33219)
0805 23:14:42.477985 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0805 23:14:42.477990 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:14:42.478104 (+ 114us) server_negotiation.cc:407] Connection header received
0805 23:14:42.478206 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:14:42.478209 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:14:42.478247 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:14:42.478342 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:14:42.479101 (+ 759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:14:42.479104 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:14:42.479107 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:14:42.479121 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:14:42.479151 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:14:42.479164 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:14:42.479254 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0805 23:14:42.479318 (+ 64us) server_negotiation.cc:299] Negotiation successful
0805 23:14:42.479358 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260805 23:14:42.482028 15044 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6683e0f439444cd19dbb6e739c8d35e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6683e0f439444cd19dbb6e739c8d35e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33219 } }
I20260805 23:14:42.482298 15044 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6683e0f439444cd19dbb6e739c8d35e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:14:42.482334 15044 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6683e0f439444cd19dbb6e739c8d35e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6683e0f439444cd19dbb6e739c8d35e6, State: Initialized, Role: FOLLOWER
I20260805 23:14:42.482718 15044 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6683e0f439444cd19dbb6e739c8d35e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6683e0f439444cd19dbb6e739c8d35e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33219 } }
I20260805 23:14:42.482831 15044 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6683e0f439444cd19dbb6e739c8d35e6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:14:42.482879 15044 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6683e0f439444cd19dbb6e739c8d35e6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:14:42.482952 15044 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6683e0f439444cd19dbb6e739c8d35e6 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:14:42.483912 15044 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6683e0f439444cd19dbb6e739c8d35e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6683e0f439444cd19dbb6e739c8d35e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33219 } }
I20260805 23:14:42.484236 15044 leader_election.cc:302] T 00000000000000000000000000000000 P 6683e0f439444cd19dbb6e739c8d35e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6683e0f439444cd19dbb6e739c8d35e6; no voters:
I20260805 23:14:42.484521 15044 leader_election.cc:288] T 00000000000000000000000000000000 P 6683e0f439444cd19dbb6e739c8d35e6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 23:14:42.484730 15054 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6683e0f439444cd19dbb6e739c8d35e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:14:42.485016 15054 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6683e0f439444cd19dbb6e739c8d35e6 [term 1 LEADER]: Becoming Leader. State: Replica: 6683e0f439444cd19dbb6e739c8d35e6, State: Running, Role: LEADER
I20260805 23:14:42.485405 15054 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6683e0f439444cd19dbb6e739c8d35e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6683e0f439444cd19dbb6e739c8d35e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33219 } }
I20260805 23:14:42.486011 15044 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6683e0f439444cd19dbb6e739c8d35e6 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:14:42.487294 15055 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6683e0f439444cd19dbb6e739c8d35e6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6683e0f439444cd19dbb6e739c8d35e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6683e0f439444cd19dbb6e739c8d35e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33219 } } }
I20260805 23:14:42.487295 15056 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6683e0f439444cd19dbb6e739c8d35e6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6683e0f439444cd19dbb6e739c8d35e6. Latest consensus state: current_term: 1 leader_uuid: "6683e0f439444cd19dbb6e739c8d35e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6683e0f439444cd19dbb6e739c8d35e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33219 } } }
I20260805 23:14:42.487452 15056 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6683e0f439444cd19dbb6e739c8d35e6 [sys.catalog]: This master's current role is: LEADER
I20260805 23:14:42.487452 15055 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6683e0f439444cd19dbb6e739c8d35e6 [sys.catalog]: This master's current role is: LEADER
I20260805 23:14:42.487890 15063 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:14:42.490769 15063 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:14:42.496824 15063 catalog_manager.cc:1347] Generated new cluster ID: 45175b9b73764eab9830cd742f033f6c
I20260805 23:14:42.496913 15063 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:14:42.521052 15063 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:14:42.521454 15063 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:14:42.528299 15063 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6683e0f439444cd19dbb6e739c8d35e6: Generated new TSK 0
I20260805 23:14:42.529232 15063 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260805 23:14:42.540279 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 14973
2026-08-05T23:14:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1461 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785971406251336-25971-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 23:14:42 dist-test-slave-ss4n krb5kdc[15075](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 23:14:42 dist-test-slave-ss4n krb5kdc[15075](info): set up 2 sockets
Aug 05 23:14:42 dist-test-slave-ss4n krb5kdc[15075](info): commencing operation
krb5kdc: starting...
W20260805 23:14:43.612768 25971 mini_kdc.cc:121] Time spent starting KDC: real 1.051s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785971406251336-25971-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785971406251336-25971-0/krb5kdc/test-user.keytab.
Aug 05 23:14:43 dist-test-slave-ss4n krb5kdc[15075](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971683, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T23:14:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:14:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785971406251336-25971-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785971406251336-25971-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260805 23:14:43.745899 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785971406251336-25971-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785971406251336-25971-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785971406251336-25971-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45495
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785971406251336-25971-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45495
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:14:43.869448 15088 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:14:43.869843 15088 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:14:43.869889 15088 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:14:43.874205 15088 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:14:43.874279 15088 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:14:43.874298 15088 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:14:43.874317 15088 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:14:43.874332 15088 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:14:43.879902 15088 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785971406251336-25971-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785971406251336-25971-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45495
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785971406251336-25971-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45495
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15088
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785971406251336-25971-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:14:43.881003 15088 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:14:43.882099 15088 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:14:43.891497 15093 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:14:43.891517 15094 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:14:43.891505 15096 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:14:43.891877 15088 server_base.cc:1034] running on GCE node
I20260805 23:14:43.892500 15088 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:14:43.893955 15088 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:14:43.895143 15088 hybrid_clock.cc:648] HybridClock initialized: now 1785971683895127 us; error 31 us; skew 500 ppm
Aug 05 23:14:43 dist-test-slave-ss4n krb5kdc[15075](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971683, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:14:43.898254 15088 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260805 23:14:43.899325 15088 webserver.cc:460] Webserver started at http://127.0.0.1:37393/ using document root <none> and password file <none>
I20260805 23:14:43.899899 15088 fs_manager.cc:359] Metadata directory not provided
I20260805 23:14:43.899963 15088 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:14:43.900136 15088 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:14:43.902501 15088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785971406251336-25971-0/minicluster-data/master-0/data/instance:
uuid: "1167add020964d9fa1f28c27fd80c817"
format_stamp: "Formatted at 2026-08-05 23:14:43 on dist-test-slave-ss4n"
server_key: "6e9c1365345828524b8615b69fb29c97"
server_key_iv: "84f356695f0631316e85990d8395ca3d"
server_key_version: "encryptionkey@0"
I20260805 23:14:43.903321 15088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785971406251336-25971-0/minicluster-data/master-0/wal/instance:
uuid: "1167add020964d9fa1f28c27fd80c817"
format_stamp: "Formatted at 2026-08-05 23:14:43 on dist-test-slave-ss4n"
server_key: "6e9c1365345828524b8615b69fb29c97"
server_key_iv: "84f356695f0631316e85990d8395ca3d"
server_key_version: "encryptionkey@0"
I20260805 23:14:43.908138 15088 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260805 23:14:43.910857 15103 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:14:43.912424 15088 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260805 23:14:43.912683 15088 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785971406251336-25971-0/minicluster-data/master-0/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785971406251336-25971-0/minicluster-data/master-0/wal
uuid: "1167add020964d9fa1f28c27fd80c817"
format_stamp: "Formatted at 2026-08-05 23:14:43 on dist-test-slave-ss4n"
server_key: "6e9c1365345828524b8615b69fb29c97"
server_key_iv: "84f356695f0631316e85990d8395ca3d"
server_key_version: "encryptionkey@0"
I20260805 23:14:43.912822 15088 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785971406251336-25971-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785971406251336-25971-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785971406251336-25971-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:14:43.931953 15088 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:14:43.935603 15088 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:14:43.935830 15088 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:14:43.943791 15088 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45495
I20260805 23:14:43.943815 15157 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45495 every 8 connection(s)
I20260805 23:14:43.944839 15088 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
I20260805 23:14:43.947824 15158 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:14:43.954034 15158 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1167add020964d9fa1f28c27fd80c817: Bootstrap starting.
I20260805 23:14:43.955063 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 15088
I20260805 23:14:43.955261 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785971406251336-25971-0/minicluster-data/master-0/wal/instance
I20260805 23:14:43.955539 25971 external_mini_cluster.cc:1428] Setting key 44b6394f1e72027861ac3f9cb598b6bd
I20260805 23:14:43.957120 15158 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1167add020964d9fa1f28c27fd80c817: Neither blocks nor log segments found. Creating new log.
I20260805 23:14:43.957968 15158 log.cc:824] T 00000000000000000000000000000000 P 1167add020964d9fa1f28c27fd80c817: Log is configured to *not* fsync() on all Append() calls
I20260805 23:14:43.960098 15158 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1167add020964d9fa1f28c27fd80c817: No bootstrap required, opened a new log
I20260805 23:14:43.964048 15158 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1167add020964d9fa1f28c27fd80c817 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1167add020964d9fa1f28c27fd80c817" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45495 } }
I20260805 23:14:43.964313 15158 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1167add020964d9fa1f28c27fd80c817 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:14:43.964404 15158 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1167add020964d9fa1f28c27fd80c817 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1167add020964d9fa1f28c27fd80c817, State: Initialized, Role: FOLLOWER
Aug 05 23:14:43 dist-test-slave-ss4n krb5kdc[15075](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971683, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260805 23:14:43.964869 15158 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1167add020964d9fa1f28c27fd80c817 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1167add020964d9fa1f28c27fd80c817" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45495 } }
I20260805 23:14:43.965086 15158 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1167add020964d9fa1f28c27fd80c817 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:14:43.965162 15158 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1167add020964d9fa1f28c27fd80c817 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:14:43.965282 15158 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1167add020964d9fa1f28c27fd80c817 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:14:43.966419 15158 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1167add020964d9fa1f28c27fd80c817 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1167add020964d9fa1f28c27fd80c817" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45495 } }
I20260805 23:14:43.966811 15158 leader_election.cc:302] T 00000000000000000000000000000000 P 1167add020964d9fa1f28c27fd80c817 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1167add020964d9fa1f28c27fd80c817; no voters:
I20260805 23:14:43.967180 15158 leader_election.cc:288] T 00000000000000000000000000000000 P 1167add020964d9fa1f28c27fd80c817 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 23:14:43.967312 15163 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1167add020964d9fa1f28c27fd80c817 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:14:43.967549 15163 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1167add020964d9fa1f28c27fd80c817 [term 1 LEADER]: Becoming Leader. State: Replica: 1167add020964d9fa1f28c27fd80c817, State: Running, Role: LEADER
I20260805 23:14:43.967967 15163 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1167add020964d9fa1f28c27fd80c817 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1167add020964d9fa1f28c27fd80c817" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45495 } }
I20260805 23:14:43.968668 15158 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1167add020964d9fa1f28c27fd80c817 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:14:43.969826 15164 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1167add020964d9fa1f28c27fd80c817 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1167add020964d9fa1f28c27fd80c817" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1167add020964d9fa1f28c27fd80c817" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45495 } } }
I20260805 23:14:43.969954 15164 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1167add020964d9fa1f28c27fd80c817 [sys.catalog]: This master's current role is: LEADER
I20260805 23:14:43.970558 15165 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1167add020964d9fa1f28c27fd80c817 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1167add020964d9fa1f28c27fd80c817. Latest consensus state: current_term: 1 leader_uuid: "1167add020964d9fa1f28c27fd80c817" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1167add020964d9fa1f28c27fd80c817" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45495 } } }
I20260805 23:14:43.970681 15165 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1167add020964d9fa1f28c27fd80c817 [sys.catalog]: This master's current role is: LEADER
I20260805 23:14:43.971271 15172 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:14:43.972530 15161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:43.956831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.245:46391 (local address 127.0.0.1:45495)
0805 23:14:43.957367 (+ 536us) server_negotiation.cc:206] Beginning negotiation
0805 23:14:43.957383 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0805 23:14:43.957421 (+ 38us) server_negotiation.cc:407] Connection header received
0805 23:14:43.958285 (+ 864us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:14:43.958305 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:14:43.958675 (+ 370us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:14:43.959030 (+ 355us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:14:43.959900 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:43.960975 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:43.961941 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:43.962473 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:43.962668 (+ 195us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:43.965777 (+ 3109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:14:43.965783 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:14:43.965799 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:14:43.965834 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:14:43.968140 (+ 2306us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:14:43.968890 (+ 750us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:14:43.968896 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:14:43.968902 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:14:43.969063 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:14:43.969552 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:14:43.969558 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:14:43.969561 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:14:43.970180 (+ 619us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:14:43.971140 (+ 960us) server_negotiation.cc:1035] Waiting for connection context
0805 23:14:43.971377 (+ 237us) server_negotiation.cc:299] Negotiation successful
0805 23:14:43.971651 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":373,"thread_start_us":161,"threads_started":1}
I20260805 23:14:43.975181 15172 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:14:43.975886 25971 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260805 23:14:43.982146 15172 catalog_manager.cc:1347] Generated new cluster ID: 83d5a4708ebb450a9300021503d30bd1
I20260805 23:14:43.982348 15172 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:14:43.988518 15161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:43.978561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.245:51217 (local address 127.0.0.1:45495)
0805 23:14:43.978682 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0805 23:14:43.978687 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:14:43.978702 (+ 15us) server_negotiation.cc:407] Connection header received
0805 23:14:43.978762 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:14:43.978765 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:14:43.978815 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:14:43.978908 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:14:43.979784 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:43.980922 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:43.982276 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:43.982633 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:43.982750 (+ 117us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:43.984520 (+ 1770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:14:43.984524 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:14:43.984527 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:14:43.984557 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:14:43.986375 (+ 1818us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:14:43.986947 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:14:43.986951 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:14:43.986953 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:14:43.987053 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:14:43.987387 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:14:43.987390 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:14:43.987392 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:14:43.987614 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:14:43.987756 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0805 23:14:43.988051 (+ 295us) server_negotiation.cc:299] Negotiation successful
0805 23:14:43.988343 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260805 23:14:44.000950 15172 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:14:44.002159 15172 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:14:44.016028 15172 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1167add020964d9fa1f28c27fd80c817: Generated new TSK 0
I20260805 23:14:44.017058 15172 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260805 23:14:44.028916 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 15088
W20260805 23:14:44.035615 15073 connection.cc:535] client connection to 127.0.0.1:45495 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45495): Connection reset by peer (error 104)
2026-08-05T23:14:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1489 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785971406251336-25971-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 23:14:44 dist-test-slave-ss4n krb5kdc[15190](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 23:14:44 dist-test-slave-ss4n krb5kdc[15190](info): set up 2 sockets
Aug 05 23:14:44 dist-test-slave-ss4n krb5kdc[15190](info): commencing operation
krb5kdc: starting...
W20260805 23:14:45.116091 25971 mini_kdc.cc:121] Time spent starting KDC: real 1.065s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785971406251336-25971-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785971406251336-25971-0/krb5kdc/test-user.keytab.
Aug 05 23:14:45 dist-test-slave-ss4n krb5kdc[15190](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971685, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T23:14:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:14:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785971406251336-25971-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785971406251336-25971-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260805 23:14:45.249382 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785971406251336-25971-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785971406251336-25971-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785971406251336-25971-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42795
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785971406251336-25971-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42795
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:14:45.361500 15203 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:14:45.361805 15203 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:14:45.361850 15203 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:14:45.366001 15203 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:14:45.366071 15203 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:14:45.366091 15203 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:14:45.366108 15203 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:14:45.366124 15203 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:14:45.371515 15203 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785971406251336-25971-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785971406251336-25971-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42795
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785971406251336-25971-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42795
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15203
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785971406251336-25971-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:14:45.372584 15203 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:14:45.373651 15203 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:14:45.382297 15209 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:14:45.382416 15203 server_base.cc:1034] running on GCE node
W20260805 23:14:45.382326 15208 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:14:45.382309 15211 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:14:45.383224 15203 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:14:45.384392 15203 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:14:45.385597 15203 hybrid_clock.cc:648] HybridClock initialized: now 1785971685385582 us; error 27 us; skew 500 ppm
Aug 05 23:14:45 dist-test-slave-ss4n krb5kdc[15190](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971685, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:14:45.388404 15203 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260805 23:14:45.389370 15203 webserver.cc:460] Webserver started at http://127.0.0.1:37335/ using document root <none> and password file <none>
I20260805 23:14:45.390007 15203 fs_manager.cc:359] Metadata directory not provided
I20260805 23:14:45.390120 15203 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:14:45.390370 15203 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:14:45.392602 15203 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785971406251336-25971-0/minicluster-data/master-0/data/instance:
uuid: "108b77a673c84055bc8d39ed79b69bdf"
format_stamp: "Formatted at 2026-08-05 23:14:45 on dist-test-slave-ss4n"
server_key: "bcfc348366f7d7350a380edcc1febceb"
server_key_iv: "47b5bfb151cc2885804e848fa76b7b5c"
server_key_version: "encryptionkey@0"
I20260805 23:14:45.393234 15203 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785971406251336-25971-0/minicluster-data/master-0/wal/instance:
uuid: "108b77a673c84055bc8d39ed79b69bdf"
format_stamp: "Formatted at 2026-08-05 23:14:45 on dist-test-slave-ss4n"
server_key: "bcfc348366f7d7350a380edcc1febceb"
server_key_iv: "47b5bfb151cc2885804e848fa76b7b5c"
server_key_version: "encryptionkey@0"
I20260805 23:14:45.397437 15203 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260805 23:14:45.400223 15218 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:14:45.401746 15203 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:14:45.401907 15203 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785971406251336-25971-0/minicluster-data/master-0/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785971406251336-25971-0/minicluster-data/master-0/wal
uuid: "108b77a673c84055bc8d39ed79b69bdf"
format_stamp: "Formatted at 2026-08-05 23:14:45 on dist-test-slave-ss4n"
server_key: "bcfc348366f7d7350a380edcc1febceb"
server_key_iv: "47b5bfb151cc2885804e848fa76b7b5c"
server_key_version: "encryptionkey@0"
I20260805 23:14:45.402053 15203 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785971406251336-25971-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785971406251336-25971-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785971406251336-25971-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:14:45.417915 15203 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:14:45.421559 15203 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:14:45.421799 15203 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:14:45.429839 15203 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42795
I20260805 23:14:45.429865 15271 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42795 every 8 connection(s)
I20260805 23:14:45.430871 15203 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
I20260805 23:14:45.433646 15272 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:14:45.436524 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 15203
I20260805 23:14:45.436719 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785971406251336-25971-0/minicluster-data/master-0/wal/instance
I20260805 23:14:45.437048 25971 external_mini_cluster.cc:1428] Setting key 96d61ea94cddfd1f201224f6ebd496c1
I20260805 23:14:45.439530 15272 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 108b77a673c84055bc8d39ed79b69bdf: Bootstrap starting.
I20260805 23:14:45.441955 15272 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 108b77a673c84055bc8d39ed79b69bdf: Neither blocks nor log segments found. Creating new log.
I20260805 23:14:45.442927 15272 log.cc:824] T 00000000000000000000000000000000 P 108b77a673c84055bc8d39ed79b69bdf: Log is configured to *not* fsync() on all Append() calls
I20260805 23:14:45.445168 15272 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 108b77a673c84055bc8d39ed79b69bdf: No bootstrap required, opened a new log
I20260805 23:14:45.446760 15275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:45.438481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.245:48489 (local address 127.0.0.1:42795)
0805 23:14:45.438927 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0805 23:14:45.438944 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0805 23:14:45.438981 (+ 37us) server_negotiation.cc:407] Connection header received
0805 23:14:45.439689 (+ 708us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:14:45.439721 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:14:45.440077 (+ 356us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:14:45.440366 (+ 289us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:14:45.440721 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:45.441931 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:45.443201 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:45.443706 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:45.443840 (+ 134us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:45.444771 (+ 931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:14:45.444776 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:14:45.444791 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:14:45.445260 (+ 469us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:14:45.445340 (+ 80us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:14:45.445367 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:14:45.445530 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0805 23:14:45.445882 (+ 352us) server_negotiation.cc:299] Negotiation successful
0805 23:14:45.446035 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":150,"threads_started":1}
I20260805 23:14:45.449283 15272 raft_consensus.cc:348] T 00000000000000000000000000000000 P 108b77a673c84055bc8d39ed79b69bdf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "108b77a673c84055bc8d39ed79b69bdf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42795 } }
I20260805 23:14:45.449664 15272 raft_consensus.cc:374] T 00000000000000000000000000000000 P 108b77a673c84055bc8d39ed79b69bdf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:14:45.449747 15272 raft_consensus.cc:729] T 00000000000000000000000000000000 P 108b77a673c84055bc8d39ed79b69bdf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 108b77a673c84055bc8d39ed79b69bdf, State: Initialized, Role: FOLLOWER
I20260805 23:14:45.450218 15272 consensus_queue.cc:260] T 00000000000000000000000000000000 P 108b77a673c84055bc8d39ed79b69bdf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "108b77a673c84055bc8d39ed79b69bdf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42795 } }
I20260805 23:14:45.450313 25971 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260805 23:14:45.450378 15272 raft_consensus.cc:388] T 00000000000000000000000000000000 P 108b77a673c84055bc8d39ed79b69bdf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:14:45.450466 15272 raft_consensus.cc:482] T 00000000000000000000000000000000 P 108b77a673c84055bc8d39ed79b69bdf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:14:45.450580 15272 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 108b77a673c84055bc8d39ed79b69bdf [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:14:45.451694 15272 raft_consensus.cc:504] T 00000000000000000000000000000000 P 108b77a673c84055bc8d39ed79b69bdf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "108b77a673c84055bc8d39ed79b69bdf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42795 } }
I20260805 23:14:45.452051 15272 leader_election.cc:302] T 00000000000000000000000000000000 P 108b77a673c84055bc8d39ed79b69bdf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 108b77a673c84055bc8d39ed79b69bdf; no voters:
I20260805 23:14:45.452409 15272 leader_election.cc:288] T 00000000000000000000000000000000 P 108b77a673c84055bc8d39ed79b69bdf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 23:14:45.452520 15281 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 108b77a673c84055bc8d39ed79b69bdf [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:14:45.452812 15281 raft_consensus.cc:686] T 00000000000000000000000000000000 P 108b77a673c84055bc8d39ed79b69bdf [term 1 LEADER]: Becoming Leader. State: Replica: 108b77a673c84055bc8d39ed79b69bdf, State: Running, Role: LEADER
I20260805 23:14:45.453143 15281 consensus_queue.cc:237] T 00000000000000000000000000000000 P 108b77a673c84055bc8d39ed79b69bdf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "108b77a673c84055bc8d39ed79b69bdf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42795 } }
I20260805 23:14:45.453812 15272 sys_catalog.cc:564] T 00000000000000000000000000000000 P 108b77a673c84055bc8d39ed79b69bdf [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:14:45.454344 15283 sys_catalog.cc:455] T 00000000000000000000000000000000 P 108b77a673c84055bc8d39ed79b69bdf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 108b77a673c84055bc8d39ed79b69bdf. Latest consensus state: current_term: 1 leader_uuid: "108b77a673c84055bc8d39ed79b69bdf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "108b77a673c84055bc8d39ed79b69bdf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42795 } } }
I20260805 23:14:45.454531 15283 sys_catalog.cc:458] T 00000000000000000000000000000000 P 108b77a673c84055bc8d39ed79b69bdf [sys.catalog]: This master's current role is: LEADER
I20260805 23:14:45.454902 15284 sys_catalog.cc:455] T 00000000000000000000000000000000 P 108b77a673c84055bc8d39ed79b69bdf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "108b77a673c84055bc8d39ed79b69bdf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "108b77a673c84055bc8d39ed79b69bdf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42795 } } }
I20260805 23:14:45.454998 15284 sys_catalog.cc:458] T 00000000000000000000000000000000 P 108b77a673c84055bc8d39ed79b69bdf [sys.catalog]: This master's current role is: LEADER
I20260805 23:14:45.458554 15275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:45.452474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.245:51039 (local address 127.0.0.1:42795)
0805 23:14:45.452588 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0805 23:14:45.452593 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:14:45.453977 (+ 1384us) server_negotiation.cc:407] Connection header received
0805 23:14:45.454180 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:14:45.454183 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:14:45.454241 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:14:45.454332 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:14:45.454805 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:45.455591 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:45.456655 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:45.457001 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:45.457112 (+ 111us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:45.458061 (+ 949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:14:45.458065 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:14:45.458068 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:14:45.458081 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:14:45.458115 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:14:45.458126 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:14:45.458234 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0805 23:14:45.458361 (+ 127us) server_negotiation.cc:299] Negotiation successful
0805 23:14:45.458394 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":43}
W20260805 23:14:45.460390 15299 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 108b77a673c84055bc8d39ed79b69bdf: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260805 23:14:45.460505 15299 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260805 23:14:45.460605 15286 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:14:45.461612 15286 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:14:45.468303 15286 catalog_manager.cc:1347] Generated new cluster ID: 85daa152f54649c5b79776ad6b9358eb
I20260805 23:14:45.468389 15286 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:14:45.498163 15286 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:14:45.499274 15286 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:14:45.506667 15286 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 108b77a673c84055bc8d39ed79b69bdf: Generated new TSK 0
I20260805 23:14:45.507513 15286 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260805 23:14:45.512319 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 15203
W20260805 23:14:45.518712 15188 connection.cc:535] client connection to 127.0.0.1:42795 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42795): Connection reset by peer (error 104)
2026-08-05T23:14:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1482 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785971406251336-25971-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 23:14:45 dist-test-slave-ss4n krb5kdc[15304](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 23:14:45 dist-test-slave-ss4n krb5kdc[15304](info): set up 2 sockets
Aug 05 23:14:45 dist-test-slave-ss4n krb5kdc[15304](info): commencing operation
krb5kdc: starting...
W20260805 23:14:46.631852 25971 mini_kdc.cc:121] Time spent starting KDC: real 1.098s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785971406251336-25971-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785971406251336-25971-0/krb5kdc/test-user.keytab.
Aug 05 23:14:46 dist-test-slave-ss4n krb5kdc[15304](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971686, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T23:14:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:14:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785971406251336-25971-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785971406251336-25971-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260805 23:14:46.760329 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785971406251336-25971-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785971406251336-25971-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785971406251336-25971-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34229
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785971406251336-25971-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34229
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:14:46.872238 15317 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:14:46.872511 15317 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:14:46.872555 15317 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:14:46.876595 15317 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:14:46.876652 15317 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:14:46.876672 15317 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:14:46.876688 15317 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:14:46.876704 15317 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:14:46.882025 15317 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785971406251336-25971-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785971406251336-25971-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34229
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785971406251336-25971-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34229
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15317
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785971406251336-25971-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:14:46.883028 15317 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:14:46.883963 15317 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:14:46.892546 15325 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:14:46.892637 15322 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:14:46.892674 15323 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:14:46.893010 15317 server_base.cc:1034] running on GCE node
I20260805 23:14:46.893748 15317 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:14:46.894968 15317 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:14:46.896181 15317 hybrid_clock.cc:648] HybridClock initialized: now 1785971686896146 us; error 52 us; skew 500 ppm
Aug 05 23:14:46 dist-test-slave-ss4n krb5kdc[15304](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971686, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:14:46.903410 15317 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260805 23:14:46.904603 15317 webserver.cc:460] Webserver started at http://127.0.0.1:36599/ using document root <none> and password file <none>
I20260805 23:14:46.905278 15317 fs_manager.cc:359] Metadata directory not provided
I20260805 23:14:46.905392 15317 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:14:46.905696 15317 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:14:46.908093 15317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785971406251336-25971-0/minicluster-data/master-0/data/instance:
uuid: "ce0cb1a9872a4c0ba208639b9ebeaf3c"
format_stamp: "Formatted at 2026-08-05 23:14:46 on dist-test-slave-ss4n"
server_key: "ee954bc5425ccf4fc84d37110ae994e9"
server_key_iv: "fc9ee18bf1b51ac33410da1f0aafea2d"
server_key_version: "encryptionkey@0"
I20260805 23:14:46.908763 15317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785971406251336-25971-0/minicluster-data/master-0/wal/instance:
uuid: "ce0cb1a9872a4c0ba208639b9ebeaf3c"
format_stamp: "Formatted at 2026-08-05 23:14:46 on dist-test-slave-ss4n"
server_key: "ee954bc5425ccf4fc84d37110ae994e9"
server_key_iv: "fc9ee18bf1b51ac33410da1f0aafea2d"
server_key_version: "encryptionkey@0"
I20260805 23:14:46.913108 15317 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260805 23:14:46.916043 15333 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:14:46.917634 15317 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:14:46.917805 15317 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785971406251336-25971-0/minicluster-data/master-0/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785971406251336-25971-0/minicluster-data/master-0/wal
uuid: "ce0cb1a9872a4c0ba208639b9ebeaf3c"
format_stamp: "Formatted at 2026-08-05 23:14:46 on dist-test-slave-ss4n"
server_key: "ee954bc5425ccf4fc84d37110ae994e9"
server_key_iv: "fc9ee18bf1b51ac33410da1f0aafea2d"
server_key_version: "encryptionkey@0"
I20260805 23:14:46.917958 15317 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785971406251336-25971-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785971406251336-25971-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785971406251336-25971-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:14:46.923326 15317 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:14:46.926940 15317 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:14:46.927189 15317 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:14:46.935302 15317 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34229
I20260805 23:14:46.935333 15385 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34229 every 8 connection(s)
I20260805 23:14:46.936326 15317 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
I20260805 23:14:46.936736 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 15317
I20260805 23:14:46.936897 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785971406251336-25971-0/minicluster-data/master-0/wal/instance
I20260805 23:14:46.937197 25971 external_mini_cluster.cc:1428] Setting key c4bf61ef6876e565e2671d3b20c3bec3
I20260805 23:14:46.940442 15386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:14:46.942596 15389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:46.938580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.245:56253 (local address 127.0.0.1:34229)
0805 23:14:46.938998 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0805 23:14:46.939014 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0805 23:14:46.939064 (+ 50us) server_negotiation.cc:407] Connection header received
0805 23:14:46.939705 (+ 641us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:14:46.939721 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:14:46.940000 (+ 279us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:14:46.940358 (+ 358us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:14:46.941361 (+ 1003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:14:46.941366 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:14:46.941377 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:14:46.941388 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:14:46.941442 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:14:46.941462 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:14:46.941648 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0805 23:14:46.941721 (+ 73us) server_negotiation.cc:299] Negotiation successful
0805 23:14:46.941887 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":114,"threads_started":1}
I20260805 23:14:46.945768 25971 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260805 23:14:46.947597 15386 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce0cb1a9872a4c0ba208639b9ebeaf3c: Bootstrap starting.
I20260805 23:14:46.950050 15389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:46.947988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.245:59615 (local address 127.0.0.1:34229)
0805 23:14:46.948155 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0805 23:14:46.948160 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:14:46.948325 (+ 165us) server_negotiation.cc:407] Connection header received
0805 23:14:46.948577 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:14:46.948580 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:14:46.948617 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:14:46.948721 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:14:46.949629 (+ 908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:14:46.949632 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:14:46.949635 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:14:46.949653 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:14:46.949679 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:14:46.949689 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:14:46.949788 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0805 23:14:46.949869 (+ 81us) server_negotiation.cc:299] Negotiation successful
0805 23:14:46.949910 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260805 23:14:46.950416 15386 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ce0cb1a9872a4c0ba208639b9ebeaf3c: Neither blocks nor log segments found. Creating new log.
I20260805 23:14:46.951304 15386 log.cc:824] T 00000000000000000000000000000000 P ce0cb1a9872a4c0ba208639b9ebeaf3c: Log is configured to *not* fsync() on all Append() calls
I20260805 23:14:46.953375 15386 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce0cb1a9872a4c0ba208639b9ebeaf3c: No bootstrap required, opened a new log
I20260805 23:14:46.957464 15386 raft_consensus.cc:348] T 00000000000000000000000000000000 P ce0cb1a9872a4c0ba208639b9ebeaf3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce0cb1a9872a4c0ba208639b9ebeaf3c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34229 } }
I20260805 23:14:46.957795 15386 raft_consensus.cc:374] T 00000000000000000000000000000000 P ce0cb1a9872a4c0ba208639b9ebeaf3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:14:46.957836 15386 raft_consensus.cc:729] T 00000000000000000000000000000000 P ce0cb1a9872a4c0ba208639b9ebeaf3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce0cb1a9872a4c0ba208639b9ebeaf3c, State: Initialized, Role: FOLLOWER
I20260805 23:14:46.958242 15386 consensus_queue.cc:260] T 00000000000000000000000000000000 P ce0cb1a9872a4c0ba208639b9ebeaf3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce0cb1a9872a4c0ba208639b9ebeaf3c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34229 } }
I20260805 23:14:46.958360 15386 raft_consensus.cc:388] T 00000000000000000000000000000000 P ce0cb1a9872a4c0ba208639b9ebeaf3c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:14:46.958403 15386 raft_consensus.cc:482] T 00000000000000000000000000000000 P ce0cb1a9872a4c0ba208639b9ebeaf3c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:14:46.958479 15386 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ce0cb1a9872a4c0ba208639b9ebeaf3c [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:14:46.959434 15386 raft_consensus.cc:504] T 00000000000000000000000000000000 P ce0cb1a9872a4c0ba208639b9ebeaf3c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce0cb1a9872a4c0ba208639b9ebeaf3c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34229 } }
I20260805 23:14:46.959753 15386 leader_election.cc:302] T 00000000000000000000000000000000 P ce0cb1a9872a4c0ba208639b9ebeaf3c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ce0cb1a9872a4c0ba208639b9ebeaf3c; no voters:
I20260805 23:14:46.960093 15386 leader_election.cc:288] T 00000000000000000000000000000000 P ce0cb1a9872a4c0ba208639b9ebeaf3c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 23:14:46.960289 15396 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ce0cb1a9872a4c0ba208639b9ebeaf3c [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:14:46.960680 15396 raft_consensus.cc:686] T 00000000000000000000000000000000 P ce0cb1a9872a4c0ba208639b9ebeaf3c [term 1 LEADER]: Becoming Leader. State: Replica: ce0cb1a9872a4c0ba208639b9ebeaf3c, State: Running, Role: LEADER
I20260805 23:14:46.961026 15396 consensus_queue.cc:237] T 00000000000000000000000000000000 P ce0cb1a9872a4c0ba208639b9ebeaf3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce0cb1a9872a4c0ba208639b9ebeaf3c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34229 } }
I20260805 23:14:46.961577 15386 sys_catalog.cc:564] T 00000000000000000000000000000000 P ce0cb1a9872a4c0ba208639b9ebeaf3c [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:14:46.963150 15398 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce0cb1a9872a4c0ba208639b9ebeaf3c [sys.catalog]: SysCatalogTable state changed. Reason: New leader ce0cb1a9872a4c0ba208639b9ebeaf3c. Latest consensus state: current_term: 1 leader_uuid: "ce0cb1a9872a4c0ba208639b9ebeaf3c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce0cb1a9872a4c0ba208639b9ebeaf3c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34229 } } }
I20260805 23:14:46.963326 15398 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce0cb1a9872a4c0ba208639b9ebeaf3c [sys.catalog]: This master's current role is: LEADER
I20260805 23:14:46.963174 15397 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce0cb1a9872a4c0ba208639b9ebeaf3c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ce0cb1a9872a4c0ba208639b9ebeaf3c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce0cb1a9872a4c0ba208639b9ebeaf3c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34229 } } }
I20260805 23:14:46.963423 15397 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce0cb1a9872a4c0ba208639b9ebeaf3c [sys.catalog]: This master's current role is: LEADER
I20260805 23:14:46.963874 15408 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:14:46.967720 15408 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:14:46.974323 15408 catalog_manager.cc:1347] Generated new cluster ID: 9bd09835875e426299c28f73fab8cdd9
I20260805 23:14:46.974418 15408 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:14:46.986408 15408 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:14:46.986796 15408 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:14:46.999331 15408 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ce0cb1a9872a4c0ba208639b9ebeaf3c: Generated new TSK 0
I20260805 23:14:47.000154 15408 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260805 23:14:47.008455 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 15317
2026-08-05T23:14:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1495 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785971406251336-25971-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 23:14:47 dist-test-slave-ss4n krb5kdc[15417](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 23:14:47 dist-test-slave-ss4n krb5kdc[15417](info): set up 2 sockets
Aug 05 23:14:47 dist-test-slave-ss4n krb5kdc[15417](info): commencing operation
krb5kdc: starting...
W20260805 23:14:48.111804 25971 mini_kdc.cc:121] Time spent starting KDC: real 1.083s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785971406251336-25971-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785971406251336-25971-0/krb5kdc/test-user.keytab.
Aug 05 23:14:48 dist-test-slave-ss4n krb5kdc[15417](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971688, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T23:14:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:14:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785971406251336-25971-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785971406251336-25971-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260805 23:14:48.248974 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785971406251336-25971-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785971406251336-25971-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785971406251336-25971-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39071
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785971406251336-25971-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39071
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:14:48.361063 15430 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:14:48.361378 15430 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:14:48.361423 15430 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:14:48.365653 15430 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:14:48.365728 15430 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:14:48.365748 15430 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:14:48.365767 15430 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:14:48.365782 15430 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:14:48.371306 15430 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785971406251336-25971-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785971406251336-25971-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39071
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785971406251336-25971-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39071
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15430
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785971406251336-25971-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:14:48.372383 15430 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:14:48.373355 15430 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:14:48.382170 15438 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:14:48.382155 15436 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:14:48.382238 15430 server_base.cc:1034] running on GCE node
W20260805 23:14:48.382150 15435 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:14:48.382972 15430 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:14:48.384071 15430 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:14:48.385258 15430 hybrid_clock.cc:648] HybridClock initialized: now 1785971688385235 us; error 36 us; skew 500 ppm
Aug 05 23:14:48 dist-test-slave-ss4n krb5kdc[15417](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971688, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:14:48.388223 15430 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260805 23:14:48.389161 15430 webserver.cc:460] Webserver started at http://127.0.0.1:41127/ using document root <none> and password file <none>
I20260805 23:14:48.389832 15430 fs_manager.cc:359] Metadata directory not provided
I20260805 23:14:48.389930 15430 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:14:48.390179 15430 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:14:48.392388 15430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785971406251336-25971-0/minicluster-data/master-0/data/instance:
uuid: "5d519e99dbe3498da141abd0c18218ce"
format_stamp: "Formatted at 2026-08-05 23:14:48 on dist-test-slave-ss4n"
server_key: "2f4f0bc36136899c39f8bef9d8d5ea03"
server_key_iv: "6ad85109f8304110cbe40799cabd01a1"
server_key_version: "encryptionkey@0"
I20260805 23:14:48.393008 15430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785971406251336-25971-0/minicluster-data/master-0/wal/instance:
uuid: "5d519e99dbe3498da141abd0c18218ce"
format_stamp: "Formatted at 2026-08-05 23:14:48 on dist-test-slave-ss4n"
server_key: "2f4f0bc36136899c39f8bef9d8d5ea03"
server_key_iv: "6ad85109f8304110cbe40799cabd01a1"
server_key_version: "encryptionkey@0"
I20260805 23:14:48.397119 15430 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260805 23:14:48.399884 15445 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:14:48.401199 15430 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260805 23:14:48.401355 15430 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785971406251336-25971-0/minicluster-data/master-0/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785971406251336-25971-0/minicluster-data/master-0/wal
uuid: "5d519e99dbe3498da141abd0c18218ce"
format_stamp: "Formatted at 2026-08-05 23:14:48 on dist-test-slave-ss4n"
server_key: "2f4f0bc36136899c39f8bef9d8d5ea03"
server_key_iv: "6ad85109f8304110cbe40799cabd01a1"
server_key_version: "encryptionkey@0"
I20260805 23:14:48.401484 15430 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785971406251336-25971-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785971406251336-25971-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785971406251336-25971-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:14:48.423321 15430 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:14:48.426896 15430 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:14:48.427146 15430 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:14:48.434841 15430 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39071
I20260805 23:14:48.434868 15499 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39071 every 8 connection(s)
I20260805 23:14:48.435943 15430 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
I20260805 23:14:48.438716 15500 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:14:48.444607 15500 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d519e99dbe3498da141abd0c18218ce: Bootstrap starting.
I20260805 23:14:48.445990 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 15430
I20260805 23:14:48.446156 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785971406251336-25971-0/minicluster-data/master-0/wal/instance
I20260805 23:14:48.446451 25971 external_mini_cluster.cc:1428] Setting key 056521e94b1ca3b613d294d3f2ffc029
I20260805 23:14:48.447593 15500 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5d519e99dbe3498da141abd0c18218ce: Neither blocks nor log segments found. Creating new log.
I20260805 23:14:48.448377 15500 log.cc:824] T 00000000000000000000000000000000 P 5d519e99dbe3498da141abd0c18218ce: Log is configured to *not* fsync() on all Append() calls
I20260805 23:14:48.451020 15500 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d519e99dbe3498da141abd0c18218ce: No bootstrap required, opened a new log
I20260805 23:14:48.455730 15500 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5d519e99dbe3498da141abd0c18218ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d519e99dbe3498da141abd0c18218ce" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39071 } }
I20260805 23:14:48.456020 15500 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5d519e99dbe3498da141abd0c18218ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:14:48.456106 15500 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5d519e99dbe3498da141abd0c18218ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d519e99dbe3498da141abd0c18218ce, State: Initialized, Role: FOLLOWER
Aug 05 23:14:48 dist-test-slave-ss4n krb5kdc[15417](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971688, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260805 23:14:48.456614 15500 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5d519e99dbe3498da141abd0c18218ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d519e99dbe3498da141abd0c18218ce" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39071 } }
I20260805 23:14:48.456736 15500 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5d519e99dbe3498da141abd0c18218ce [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:14:48.456820 15500 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5d519e99dbe3498da141abd0c18218ce [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:14:48.456919 15500 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5d519e99dbe3498da141abd0c18218ce [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:14:48.457973 15500 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5d519e99dbe3498da141abd0c18218ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d519e99dbe3498da141abd0c18218ce" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39071 } }
I20260805 23:14:48.458329 15500 leader_election.cc:302] T 00000000000000000000000000000000 P 5d519e99dbe3498da141abd0c18218ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5d519e99dbe3498da141abd0c18218ce; no voters:
I20260805 23:14:48.458709 15500 leader_election.cc:288] T 00000000000000000000000000000000 P 5d519e99dbe3498da141abd0c18218ce [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 23:14:48.458773 15505 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5d519e99dbe3498da141abd0c18218ce [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:14:48.458981 15505 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5d519e99dbe3498da141abd0c18218ce [term 1 LEADER]: Becoming Leader. State: Replica: 5d519e99dbe3498da141abd0c18218ce, State: Running, Role: LEADER
I20260805 23:14:48.459340 15505 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5d519e99dbe3498da141abd0c18218ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d519e99dbe3498da141abd0c18218ce" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39071 } }
I20260805 23:14:48.460098 15500 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5d519e99dbe3498da141abd0c18218ce [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:14:48.460557 15505 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d519e99dbe3498da141abd0c18218ce [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5d519e99dbe3498da141abd0c18218ce. Latest consensus state: current_term: 1 leader_uuid: "5d519e99dbe3498da141abd0c18218ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d519e99dbe3498da141abd0c18218ce" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39071 } } }
I20260805 23:14:48.461357 15505 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d519e99dbe3498da141abd0c18218ce [sys.catalog]: This master's current role is: LEADER
I20260805 23:14:48.461243 15506 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d519e99dbe3498da141abd0c18218ce [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5d519e99dbe3498da141abd0c18218ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d519e99dbe3498da141abd0c18218ce" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39071 } } }
I20260805 23:14:48.461714 15506 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d519e99dbe3498da141abd0c18218ce [sys.catalog]: This master's current role is: LEADER
I20260805 23:14:48.462545 15515 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:14:48.463200 15503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:48.447727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.245:50041 (local address 127.0.0.1:39071)
0805 23:14:48.448211 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0805 23:14:48.448228 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0805 23:14:48.448263 (+ 35us) server_negotiation.cc:407] Connection header received
0805 23:14:48.449194 (+ 931us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:14:48.449223 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:14:48.449710 (+ 487us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:14:48.449999 (+ 289us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:14:48.450929 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:48.451961 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:48.453167 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:48.453757 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:48.453884 (+ 127us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:48.457068 (+ 3184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:14:48.457075 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:14:48.457096 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:14:48.457125 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:14:48.459155 (+ 2030us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:14:48.459798 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:14:48.459804 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:14:48.459810 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:14:48.459929 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:14:48.460329 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:14:48.460332 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:14:48.460333 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:14:48.460873 (+ 540us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:14:48.461868 (+ 995us) server_negotiation.cc:1035] Waiting for connection context
0805 23:14:48.462061 (+ 193us) server_negotiation.cc:299] Negotiation successful
0805 23:14:48.462325 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":170,"threads_started":1}
I20260805 23:14:48.466228 15515 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:14:48.466614 25971 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260805 23:14:48.472782 15515 catalog_manager.cc:1347] Generated new cluster ID: c2276cf99119471fb17b6a11cc5d0cec
I20260805 23:14:48.472862 15515 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:14:48.479630 15503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:48.468847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.245:42215 (local address 127.0.0.1:39071)
0805 23:14:48.468975 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0805 23:14:48.468981 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:14:48.469342 (+ 361us) server_negotiation.cc:407] Connection header received
0805 23:14:48.469514 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:14:48.469518 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:14:48.469567 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:14:48.469663 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:14:48.470375 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:48.471522 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:48.473619 (+ 2097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:48.473977 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:48.474076 (+ 99us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:48.475875 (+ 1799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:14:48.475879 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:14:48.475882 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:14:48.475909 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:14:48.477692 (+ 1783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:14:48.478258 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:14:48.478262 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:14:48.478264 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:14:48.478351 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:14:48.478697 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:14:48.478700 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:14:48.478702 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:14:48.478911 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:14:48.479060 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0805 23:14:48.479359 (+ 299us) server_negotiation.cc:299] Negotiation successful
0805 23:14:48.479463 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260805 23:14:48.481678 15515 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:14:48.482890 15515 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:14:48.504175 15515 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5d519e99dbe3498da141abd0c18218ce: Generated new TSK 0
I20260805 23:14:48.505086 15515 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260805 23:14:48.513484 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 15430
W20260805 23:14:48.519773 15415 connection.cc:535] client connection to 127.0.0.1:39071 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39071): Connection reset by peer (error 104)
2026-08-05T23:14:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1505 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785971406251336-25971-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 23:14:48 dist-test-slave-ss4n krb5kdc[15532](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 23:14:48 dist-test-slave-ss4n krb5kdc[15532](info): set up 2 sockets
Aug 05 23:14:48 dist-test-slave-ss4n krb5kdc[15532](info): commencing operation
krb5kdc: starting...
W20260805 23:14:49.579847 25971 mini_kdc.cc:121] Time spent starting KDC: real 1.045s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785971406251336-25971-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785971406251336-25971-0/krb5kdc/test-user.keytab.
Aug 05 23:14:49 dist-test-slave-ss4n krb5kdc[15532](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971689, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T23:14:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:14:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785971406251336-25971-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785971406251336-25971-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260805 23:14:49.709755 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785971406251336-25971-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785971406251336-25971-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785971406251336-25971-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36971
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785971406251336-25971-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39119
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36971
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:14:49.818511 15545 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:14:49.818775 15545 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:14:49.818815 15545 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:14:49.822820 15545 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:14:49.822882 15545 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:14:49.822902 15545 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:14:49.822918 15545 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:14:49.822933 15545 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:14:49.828102 15545 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39119
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785971406251336-25971-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785971406251336-25971-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36971
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785971406251336-25971-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36971
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15545
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785971406251336-25971-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:14:49.829116 15545 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:14:49.830183 15545 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:14:49.838671 15553 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:14:49.838788 15545 server_base.cc:1034] running on GCE node
W20260805 23:14:49.838681 15550 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:14:49.839035 15551 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:14:49.839620 15545 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:14:49.840720 15545 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:14:49.841904 15545 hybrid_clock.cc:648] HybridClock initialized: now 1785971689841885 us; error 37 us; skew 500 ppm
Aug 05 23:14:49 dist-test-slave-ss4n krb5kdc[15532](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971689, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:14:49.844702 15545 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260805 23:14:49.845752 15545 webserver.cc:460] Webserver started at http://127.0.0.1:45743/ using document root <none> and password file <none>
I20260805 23:14:49.846310 15545 fs_manager.cc:359] Metadata directory not provided
I20260805 23:14:49.846400 15545 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:14:49.846607 15545 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:14:49.848654 15545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785971406251336-25971-0/minicluster-data/master-0/data/instance:
uuid: "0205bc326e2e44a48ea99f80cef62a9b"
format_stamp: "Formatted at 2026-08-05 23:14:49 on dist-test-slave-ss4n"
server_key: "5a0a648d5c20a02f349672238072a95a"
server_key_iv: "5a5233fd7aea9f337c2b80348afe66ca"
server_key_version: "encryptionkey@0"
I20260805 23:14:49.849231 15545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785971406251336-25971-0/minicluster-data/master-0/wal/instance:
uuid: "0205bc326e2e44a48ea99f80cef62a9b"
format_stamp: "Formatted at 2026-08-05 23:14:49 on dist-test-slave-ss4n"
server_key: "5a0a648d5c20a02f349672238072a95a"
server_key_iv: "5a5233fd7aea9f337c2b80348afe66ca"
server_key_version: "encryptionkey@0"
I20260805 23:14:49.853108 15545 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260805 23:14:49.855795 15560 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:14:49.857087 15545 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260805 23:14:49.857241 15545 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785971406251336-25971-0/minicluster-data/master-0/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785971406251336-25971-0/minicluster-data/master-0/wal
uuid: "0205bc326e2e44a48ea99f80cef62a9b"
format_stamp: "Formatted at 2026-08-05 23:14:49 on dist-test-slave-ss4n"
server_key: "5a0a648d5c20a02f349672238072a95a"
server_key_iv: "5a5233fd7aea9f337c2b80348afe66ca"
server_key_version: "encryptionkey@0"
I20260805 23:14:49.857364 15545 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785971406251336-25971-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785971406251336-25971-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785971406251336-25971-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:14:49.876291 15545 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:14:49.879382 15545 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:14:49.879585 15545 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:14:49.887141 15545 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36971
I20260805 23:14:49.887158 15613 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36971 every 8 connection(s)
I20260805 23:14:49.888111 15545 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
I20260805 23:14:49.890914 15615 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:14:49.896701 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 15545
I20260805 23:14:49.896867 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785971406251336-25971-0/minicluster-data/master-0/wal/instance
I20260805 23:14:49.897198 25971 external_mini_cluster.cc:1428] Setting key 70204ea7760a8a051ebc5809aa588370
I20260805 23:14:49.897261 15615 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0205bc326e2e44a48ea99f80cef62a9b: Bootstrap starting.
I20260805 23:14:49.900128 15615 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0205bc326e2e44a48ea99f80cef62a9b: Neither blocks nor log segments found. Creating new log.
I20260805 23:14:49.900957 15615 log.cc:824] T 00000000000000000000000000000000 P 0205bc326e2e44a48ea99f80cef62a9b: Log is configured to *not* fsync() on all Append() calls
I20260805 23:14:49.903079 15615 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0205bc326e2e44a48ea99f80cef62a9b: No bootstrap required, opened a new log
I20260805 23:14:49.906617 15618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:49.898688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.245:41609 (local address 127.0.0.1:36971)
0805 23:14:49.899184 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0805 23:14:49.899199 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0805 23:14:49.899234 (+ 35us) server_negotiation.cc:407] Connection header received
0805 23:14:49.899876 (+ 642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:14:49.899893 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:14:49.900282 (+ 389us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:14:49.900621 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:14:49.901064 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:49.902302 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:49.903246 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:49.903712 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:49.903888 (+ 176us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:49.904886 (+ 998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:14:49.904891 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:14:49.904903 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:14:49.905298 (+ 395us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:14:49.905354 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:14:49.905377 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:14:49.905559 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0805 23:14:49.905746 (+ 187us) server_negotiation.cc:299] Negotiation successful
0805 23:14:49.905893 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":164,"threads_started":1}
I20260805 23:14:49.907564 15615 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0205bc326e2e44a48ea99f80cef62a9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0205bc326e2e44a48ea99f80cef62a9b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36971 } }
I20260805 23:14:49.907904 15615 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0205bc326e2e44a48ea99f80cef62a9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:14:49.907996 15615 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0205bc326e2e44a48ea99f80cef62a9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0205bc326e2e44a48ea99f80cef62a9b, State: Initialized, Role: FOLLOWER
I20260805 23:14:49.908499 15615 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0205bc326e2e44a48ea99f80cef62a9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0205bc326e2e44a48ea99f80cef62a9b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36971 } }
I20260805 23:14:49.908710 15615 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0205bc326e2e44a48ea99f80cef62a9b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:14:49.908787 15615 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0205bc326e2e44a48ea99f80cef62a9b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:14:49.908893 15615 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0205bc326e2e44a48ea99f80cef62a9b [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:14:49.909972 25971 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260805 23:14:49.910202 15615 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0205bc326e2e44a48ea99f80cef62a9b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0205bc326e2e44a48ea99f80cef62a9b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36971 } }
I20260805 23:14:49.910557 15615 leader_election.cc:302] T 00000000000000000000000000000000 P 0205bc326e2e44a48ea99f80cef62a9b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0205bc326e2e44a48ea99f80cef62a9b; no voters:
I20260805 23:14:49.910913 15615 leader_election.cc:288] T 00000000000000000000000000000000 P 0205bc326e2e44a48ea99f80cef62a9b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 23:14:49.911016 15620 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0205bc326e2e44a48ea99f80cef62a9b [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:14:49.911216 15620 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0205bc326e2e44a48ea99f80cef62a9b [term 1 LEADER]: Becoming Leader. State: Replica: 0205bc326e2e44a48ea99f80cef62a9b, State: Running, Role: LEADER
I20260805 23:14:49.911610 15620 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0205bc326e2e44a48ea99f80cef62a9b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0205bc326e2e44a48ea99f80cef62a9b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36971 } }
I20260805 23:14:49.912408 15615 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0205bc326e2e44a48ea99f80cef62a9b [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:14:49.912977 15620 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0205bc326e2e44a48ea99f80cef62a9b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0205bc326e2e44a48ea99f80cef62a9b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0205bc326e2e44a48ea99f80cef62a9b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36971 } } }
I20260805 23:14:49.913168 15620 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0205bc326e2e44a48ea99f80cef62a9b [sys.catalog]: This master's current role is: LEADER
I20260805 23:14:49.913511 15620 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0205bc326e2e44a48ea99f80cef62a9b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0205bc326e2e44a48ea99f80cef62a9b. Latest consensus state: current_term: 1 leader_uuid: "0205bc326e2e44a48ea99f80cef62a9b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0205bc326e2e44a48ea99f80cef62a9b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36971 } } }
I20260805 23:14:49.913691 15620 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0205bc326e2e44a48ea99f80cef62a9b [sys.catalog]: This master's current role is: LEADER
I20260805 23:14:49.915791 15632 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:14:49.918723 15618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:49.912126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.245:51095 (local address 127.0.0.1:36971)
0805 23:14:49.912268 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0805 23:14:49.912277 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0805 23:14:49.912588 (+ 311us) server_negotiation.cc:407] Connection header received
0805 23:14:49.912740 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:14:49.912743 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:14:49.912790 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:14:49.912865 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:14:49.913278 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:49.914191 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:49.915527 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:49.915999 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:49.916162 (+ 163us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:49.917285 (+ 1123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:14:49.917326 (+ 41us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:14:49.917333 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:14:49.917354 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:14:49.917431 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:14:49.917466 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:14:49.918243 (+ 777us) server_negotiation.cc:1035] Waiting for connection context
0805 23:14:49.918355 (+ 112us) server_negotiation.cc:299] Negotiation successful
0805 23:14:49.918426 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260805 23:14:49.920022 15632 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:14:49.926901 15632 catalog_manager.cc:1347] Generated new cluster ID: 807cd8dc85374885b08973ea64d555df
I20260805 23:14:49.926998 15632 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:14:49.942041 15632 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:14:49.943120 15632 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:14:49.948477 15632 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0205bc326e2e44a48ea99f80cef62a9b: Generated new TSK 0
I20260805 23:14:49.949159 15632 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260805 23:14:49.954310 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 15545
W20260805 23:14:49.961001 15530 connection.cc:535] client connection to 127.0.0.1:36971 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36971): Connection reset by peer (error 104)
2026-08-05T23:14:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1440 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785971406251336-25971-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 23:14:50 dist-test-slave-ss4n krb5kdc[15646](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 23:14:50 dist-test-slave-ss4n krb5kdc[15646](info): set up 2 sockets
Aug 05 23:14:50 dist-test-slave-ss4n krb5kdc[15646](info): commencing operation
krb5kdc: starting...
W20260805 23:14:51.004030 25971 mini_kdc.cc:121] Time spent starting KDC: real 1.028s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785971406251336-25971-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785971406251336-25971-0/krb5kdc/test-user.keytab.
Aug 05 23:14:51 dist-test-slave-ss4n krb5kdc[15646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971691, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T23:14:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:14:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785971406251336-25971-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785971406251336-25971-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260805 23:14:51.137171 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785971406251336-25971-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785971406251336-25971-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785971406251336-25971-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34641
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785971406251336-25971-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34641
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:14:51.256011 15660 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:14:51.256289 15660 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:14:51.256332 15660 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:14:51.260576 15660 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:14:51.260661 15660 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:14:51.260681 15660 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:14:51.260699 15660 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:14:51.260763 15660 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:14:51.266283 15660 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785971406251336-25971-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785971406251336-25971-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34641
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785971406251336-25971-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34641
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15660
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785971406251336-25971-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:14:51.267429 15660 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:14:51.268492 15660 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:14:51.277407 15668 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:14:51.277669 15665 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:14:51.277590 15666 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:14:51.277590 15660 server_base.cc:1034] running on GCE node
I20260805 23:14:51.278332 15660 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:14:51.279464 15660 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:14:51.280665 15660 hybrid_clock.cc:648] HybridClock initialized: now 1785971691280620 us; error 35 us; skew 500 ppm
Aug 05 23:14:51 dist-test-slave-ss4n krb5kdc[15646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971691, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:14:51.283567 15660 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260805 23:14:51.284446 15660 webserver.cc:460] Webserver started at http://127.0.0.1:43233/ using document root <none> and password file <none>
I20260805 23:14:51.284999 15660 fs_manager.cc:359] Metadata directory not provided
I20260805 23:14:51.285058 15660 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:14:51.285266 15660 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:14:51.287323 15660 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785971406251336-25971-0/minicluster-data/master-0/data/instance:
uuid: "324edf086fca4ba887525d37edd665b4"
format_stamp: "Formatted at 2026-08-05 23:14:51 on dist-test-slave-ss4n"
server_key: "a7c77f01c1509bc4c19eab3aa7cc495f"
server_key_iv: "42b6ddba350ec1f73aa1ef7d8e206b50"
server_key_version: "encryptionkey@0"
I20260805 23:14:51.287880 15660 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785971406251336-25971-0/minicluster-data/master-0/wal/instance:
uuid: "324edf086fca4ba887525d37edd665b4"
format_stamp: "Formatted at 2026-08-05 23:14:51 on dist-test-slave-ss4n"
server_key: "a7c77f01c1509bc4c19eab3aa7cc495f"
server_key_iv: "42b6ddba350ec1f73aa1ef7d8e206b50"
server_key_version: "encryptionkey@0"
I20260805 23:14:51.291627 15660 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260805 23:14:51.294286 15675 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:14:51.295722 15660 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260805 23:14:51.295881 15660 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785971406251336-25971-0/minicluster-data/master-0/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785971406251336-25971-0/minicluster-data/master-0/wal
uuid: "324edf086fca4ba887525d37edd665b4"
format_stamp: "Formatted at 2026-08-05 23:14:51 on dist-test-slave-ss4n"
server_key: "a7c77f01c1509bc4c19eab3aa7cc495f"
server_key_iv: "42b6ddba350ec1f73aa1ef7d8e206b50"
server_key_version: "encryptionkey@0"
I20260805 23:14:51.296011 15660 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785971406251336-25971-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785971406251336-25971-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785971406251336-25971-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:14:51.300936 15660 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:14:51.304611 15660 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:14:51.304831 15660 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:14:51.312664 15660 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34641
I20260805 23:14:51.312726 15728 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34641 every 8 connection(s)
I20260805 23:14:51.313794 15660 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
I20260805 23:14:51.314213 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 15660
I20260805 23:14:51.314386 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785971406251336-25971-0/minicluster-data/master-0/wal/instance
I20260805 23:14:51.314710 25971 external_mini_cluster.cc:1428] Setting key 8ded552beb7ab1eeebb481108de66375
I20260805 23:14:51.317417 15729 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:14:51.320204 15732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:51.316308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.245:53017 (local address 127.0.0.1:34641)
0805 23:14:51.316779 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0805 23:14:51.316795 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0805 23:14:51.316829 (+ 34us) server_negotiation.cc:407] Connection header received
0805 23:14:51.317461 (+ 632us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:14:51.317475 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:14:51.317838 (+ 363us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:14:51.318175 (+ 337us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:14:51.319041 (+ 866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:14:51.319046 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:14:51.319067 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:14:51.319079 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:14:51.319133 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:14:51.319152 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:14:51.319302 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0805 23:14:51.319373 (+ 71us) server_negotiation.cc:299] Negotiation successful
0805 23:14:51.319522 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":169,"threads_started":1}
I20260805 23:14:51.323057 25971 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260805 23:14:51.324271 15729 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 324edf086fca4ba887525d37edd665b4: Bootstrap starting.
I20260805 23:14:51.327090 15732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:51.325200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.245:56739 (local address 127.0.0.1:34641)
0805 23:14:51.325317 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0805 23:14:51.325320 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 23:14:51.325607 (+ 287us) server_negotiation.cc:407] Connection header received
0805 23:14:51.325758 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:14:51.325760 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:14:51.325794 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:14:51.325853 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:14:51.326631 (+ 778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:14:51.326634 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:14:51.326636 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:14:51.326647 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:14:51.326672 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:14:51.326682 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:14:51.326752 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0805 23:14:51.326883 (+ 131us) server_negotiation.cc:299] Negotiation successful
0805 23:14:51.326919 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260805 23:14:51.327549 15729 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 324edf086fca4ba887525d37edd665b4: Neither blocks nor log segments found. Creating new log.
I20260805 23:14:51.328445 15729 log.cc:824] T 00000000000000000000000000000000 P 324edf086fca4ba887525d37edd665b4: Log is configured to *not* fsync() on all Append() calls
I20260805 23:14:51.330605 15729 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 324edf086fca4ba887525d37edd665b4: No bootstrap required, opened a new log
I20260805 23:14:51.334754 15729 raft_consensus.cc:348] T 00000000000000000000000000000000 P 324edf086fca4ba887525d37edd665b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "324edf086fca4ba887525d37edd665b4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34641 } }
I20260805 23:14:51.335003 15729 raft_consensus.cc:374] T 00000000000000000000000000000000 P 324edf086fca4ba887525d37edd665b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:14:51.335052 15729 raft_consensus.cc:729] T 00000000000000000000000000000000 P 324edf086fca4ba887525d37edd665b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 324edf086fca4ba887525d37edd665b4, State: Initialized, Role: FOLLOWER
I20260805 23:14:51.335465 15729 consensus_queue.cc:260] T 00000000000000000000000000000000 P 324edf086fca4ba887525d37edd665b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "324edf086fca4ba887525d37edd665b4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34641 } }
I20260805 23:14:51.335585 15729 raft_consensus.cc:388] T 00000000000000000000000000000000 P 324edf086fca4ba887525d37edd665b4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:14:51.335628 15729 raft_consensus.cc:482] T 00000000000000000000000000000000 P 324edf086fca4ba887525d37edd665b4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:14:51.335704 15729 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 324edf086fca4ba887525d37edd665b4 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:14:51.336655 15729 raft_consensus.cc:504] T 00000000000000000000000000000000 P 324edf086fca4ba887525d37edd665b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "324edf086fca4ba887525d37edd665b4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34641 } }
I20260805 23:14:51.336958 15729 leader_election.cc:302] T 00000000000000000000000000000000 P 324edf086fca4ba887525d37edd665b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 324edf086fca4ba887525d37edd665b4; no voters:
I20260805 23:14:51.337255 15729 leader_election.cc:288] T 00000000000000000000000000000000 P 324edf086fca4ba887525d37edd665b4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 23:14:51.337352 15739 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 324edf086fca4ba887525d37edd665b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:14:51.337664 15739 raft_consensus.cc:686] T 00000000000000000000000000000000 P 324edf086fca4ba887525d37edd665b4 [term 1 LEADER]: Becoming Leader. State: Replica: 324edf086fca4ba887525d37edd665b4, State: Running, Role: LEADER
I20260805 23:14:51.338042 15739 consensus_queue.cc:237] T 00000000000000000000000000000000 P 324edf086fca4ba887525d37edd665b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "324edf086fca4ba887525d37edd665b4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34641 } }
I20260805 23:14:51.338650 15729 sys_catalog.cc:564] T 00000000000000000000000000000000 P 324edf086fca4ba887525d37edd665b4 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:14:51.339766 15741 sys_catalog.cc:455] T 00000000000000000000000000000000 P 324edf086fca4ba887525d37edd665b4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 324edf086fca4ba887525d37edd665b4. Latest consensus state: current_term: 1 leader_uuid: "324edf086fca4ba887525d37edd665b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "324edf086fca4ba887525d37edd665b4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34641 } } }
I20260805 23:14:51.339766 15740 sys_catalog.cc:455] T 00000000000000000000000000000000 P 324edf086fca4ba887525d37edd665b4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "324edf086fca4ba887525d37edd665b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "324edf086fca4ba887525d37edd665b4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34641 } } }
I20260805 23:14:51.339978 15741 sys_catalog.cc:458] T 00000000000000000000000000000000 P 324edf086fca4ba887525d37edd665b4 [sys.catalog]: This master's current role is: LEADER
I20260805 23:14:51.339978 15740 sys_catalog.cc:458] T 00000000000000000000000000000000 P 324edf086fca4ba887525d37edd665b4 [sys.catalog]: This master's current role is: LEADER
I20260805 23:14:51.340466 15748 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:14:51.343775 15748 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:14:51.350533 15748 catalog_manager.cc:1347] Generated new cluster ID: 718ef346943c495f9929b474654e5033
I20260805 23:14:51.350665 15748 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:14:51.366020 15748 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:14:51.366403 15748 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:14:51.374572 15748 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 324edf086fca4ba887525d37edd665b4: Generated new TSK 0
I20260805 23:14:51.375331 15748 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260805 23:14:51.379480 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 15660
2026-08-05T23:14:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1424 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785971406251336-25971-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 23:14:51 dist-test-slave-ss4n krb5kdc[15761](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 23:14:51 dist-test-slave-ss4n krb5kdc[15761](info): set up 2 sockets
Aug 05 23:14:51 dist-test-slave-ss4n krb5kdc[15761](info): commencing operation
krb5kdc: starting...
W20260805 23:14:52.500200 25971 mini_kdc.cc:121] Time spent starting KDC: real 1.100s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785971406251336-25971-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785971406251336-25971-0/krb5kdc/test-user.keytab.
Aug 05 23:14:52 dist-test-slave-ss4n krb5kdc[15761](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971692, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T23:14:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:14:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.92.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.92.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.92.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab.
Entry for principal kudu/127.25.92.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab.
I20260805 23:14:52.632160 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785971406251336-25971-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785971406251336-25971-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785971406251336-25971-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.254:35085
--webserver_interface=127.25.92.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab
--principal=kudu/127.25.92.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:36897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.92.254:35085
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:14:52.749358 15775 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:14:52.749766 15775 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:14:52.749857 15775 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:14:52.753943 15775 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:14:52.754042 15775 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:14:52.754082 15775 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:14:52.754136 15775 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:14:52.754170 15775 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:14:52.759605 15775 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:36897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785971406251336-25971-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785971406251336-25971-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.92.254:35085
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab
--principal=kudu/127.25.92.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.254:35085
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.92.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15775
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785971406251336-25971-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:14:52.760802 15775 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:14:52.761942 15775 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:14:52.770841 15780 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:14:52.770853 15781 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:14:52.771054 15775 server_base.cc:1034] running on GCE node
W20260805 23:14:52.770853 15783 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:14:52.771984 15775 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:14:52.773240 15775 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:14:52.774446 15775 hybrid_clock.cc:648] HybridClock initialized: now 1785971692774428 us; error 64 us; skew 500 ppm
Aug 05 23:14:52 dist-test-slave-ss4n krb5kdc[15761](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971692, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:14:52.782845 15775 init.cc:377] Logged in from keytab as kudu/127.25.92.254@KRBTEST.COM (short username kudu)
I20260805 23:14:52.784328 15775 webserver.cc:460] Webserver started at http://127.25.92.254:40327/ using document root <none> and password file <none>
I20260805 23:14:52.785120 15775 fs_manager.cc:359] Metadata directory not provided
I20260805 23:14:52.785225 15775 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:14:52.785543 15775 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:14:52.788662 15775 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785971406251336-25971-0/minicluster-data/master-0/data/instance:
uuid: "27fda1fe5d744f918107ab21d3cadff1"
format_stamp: "Formatted at 2026-08-05 23:14:52 on dist-test-slave-ss4n"
server_key: "16be76b54b10e924e13266633fb67080"
server_key_iv: "73c4adb3939f96a91769b2bd99862807"
server_key_version: "encryptionkey@0"
I20260805 23:14:52.789470 15775 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785971406251336-25971-0/minicluster-data/master-0/wal/instance:
uuid: "27fda1fe5d744f918107ab21d3cadff1"
format_stamp: "Formatted at 2026-08-05 23:14:52 on dist-test-slave-ss4n"
server_key: "16be76b54b10e924e13266633fb67080"
server_key_iv: "73c4adb3939f96a91769b2bd99862807"
server_key_version: "encryptionkey@0"
I20260805 23:14:52.794113 15775 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260805 23:14:52.796820 15790 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:14:52.798276 15775 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:14:52.798439 15775 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785971406251336-25971-0/minicluster-data/master-0/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785971406251336-25971-0/minicluster-data/master-0/wal
uuid: "27fda1fe5d744f918107ab21d3cadff1"
format_stamp: "Formatted at 2026-08-05 23:14:52 on dist-test-slave-ss4n"
server_key: "16be76b54b10e924e13266633fb67080"
server_key_iv: "73c4adb3939f96a91769b2bd99862807"
server_key_version: "encryptionkey@0"
I20260805 23:14:52.798566 15775 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785971406251336-25971-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785971406251336-25971-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785971406251336-25971-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:14:52.824633 15775 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:14:52.827930 15775 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:14:52.828166 15775 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:14:52.835919 15775 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.254:35085
I20260805 23:14:52.835940 15844 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.254:35085 every 8 connection(s)
I20260805 23:14:52.836874 15775 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
I20260805 23:14:52.839783 15845 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:14:52.839951 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 15775
I20260805 23:14:52.840111 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785971406251336-25971-0/minicluster-data/master-0/wal/instance
I20260805 23:14:52.840382 25971 external_mini_cluster.cc:1428] Setting key 3c945c9f613ac30ecb184c49159c5aaa
I20260805 23:14:52.846423 15845 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27fda1fe5d744f918107ab21d3cadff1: Bootstrap starting.
I20260805 23:14:52.848901 15845 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 27fda1fe5d744f918107ab21d3cadff1: Neither blocks nor log segments found. Creating new log.
Aug 05 23:14:52 dist-test-slave-ss4n krb5kdc[15761](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971692, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM
I20260805 23:14:52.849936 15845 log.cc:824] T 00000000000000000000000000000000 P 27fda1fe5d744f918107ab21d3cadff1: Log is configured to *not* fsync() on all Append() calls
I20260805 23:14:52.852212 15845 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27fda1fe5d744f918107ab21d3cadff1: No bootstrap required, opened a new log
I20260805 23:14:52.855480 15848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:52.841832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40137 (local address 127.25.92.254:35085)
0805 23:14:52.842298 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0805 23:14:52.842313 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0805 23:14:52.842350 (+ 37us) server_negotiation.cc:407] Connection header received
0805 23:14:52.843064 (+ 714us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:14:52.843093 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:14:52.843425 (+ 332us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:14:52.843773 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:14:52.844570 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:52.845647 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:52.846495 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:52.847014 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:52.847206 (+ 192us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:52.850484 (+ 3278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:14:52.850490 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:14:52.850505 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:14:52.850592 (+ 87us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:14:52.852464 (+ 1872us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:14:52.853012 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:14:52.853017 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:14:52.853022 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:14:52.853135 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:14:52.853521 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:14:52.853525 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:14:52.853527 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:14:52.853933 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:14:52.854169 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0805 23:14:52.854493 (+ 324us) server_negotiation.cc:299] Negotiation successful
0805 23:14:52.854724 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":149,"threads_started":1}
I20260805 23:14:52.856096 15845 raft_consensus.cc:348] T 00000000000000000000000000000000 P 27fda1fe5d744f918107ab21d3cadff1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27fda1fe5d744f918107ab21d3cadff1" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 35085 } }
I20260805 23:14:52.856369 15845 raft_consensus.cc:374] T 00000000000000000000000000000000 P 27fda1fe5d744f918107ab21d3cadff1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:14:52.856426 15845 raft_consensus.cc:729] T 00000000000000000000000000000000 P 27fda1fe5d744f918107ab21d3cadff1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27fda1fe5d744f918107ab21d3cadff1, State: Initialized, Role: FOLLOWER
I20260805 23:14:52.856841 15845 consensus_queue.cc:260] T 00000000000000000000000000000000 P 27fda1fe5d744f918107ab21d3cadff1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27fda1fe5d744f918107ab21d3cadff1" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 35085 } }
I20260805 23:14:52.856961 15845 raft_consensus.cc:388] T 00000000000000000000000000000000 P 27fda1fe5d744f918107ab21d3cadff1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:14:52.857052 15845 raft_consensus.cc:482] T 00000000000000000000000000000000 P 27fda1fe5d744f918107ab21d3cadff1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:14:52.857159 15845 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 27fda1fe5d744f918107ab21d3cadff1 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:14:52.858355 15845 raft_consensus.cc:504] T 00000000000000000000000000000000 P 27fda1fe5d744f918107ab21d3cadff1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27fda1fe5d744f918107ab21d3cadff1" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 35085 } }
I20260805 23:14:52.858698 25971 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260805 23:14:52.858731 15845 leader_election.cc:302] T 00000000000000000000000000000000 P 27fda1fe5d744f918107ab21d3cadff1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 27fda1fe5d744f918107ab21d3cadff1; no voters:
I20260805 23:14:52.859141 15845 leader_election.cc:288] T 00000000000000000000000000000000 P 27fda1fe5d744f918107ab21d3cadff1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 23:14:52.859221 15850 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 27fda1fe5d744f918107ab21d3cadff1 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:14:52.859419 15850 raft_consensus.cc:686] T 00000000000000000000000000000000 P 27fda1fe5d744f918107ab21d3cadff1 [term 1 LEADER]: Becoming Leader. State: Replica: 27fda1fe5d744f918107ab21d3cadff1, State: Running, Role: LEADER
I20260805 23:14:52.859784 15850 consensus_queue.cc:237] T 00000000000000000000000000000000 P 27fda1fe5d744f918107ab21d3cadff1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27fda1fe5d744f918107ab21d3cadff1" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 35085 } }
I20260805 23:14:52.860471 15845 sys_catalog.cc:564] T 00000000000000000000000000000000 P 27fda1fe5d744f918107ab21d3cadff1 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:14:52.861527 15853 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27fda1fe5d744f918107ab21d3cadff1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "27fda1fe5d744f918107ab21d3cadff1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27fda1fe5d744f918107ab21d3cadff1" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 35085 } } }
I20260805 23:14:52.861699 15853 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27fda1fe5d744f918107ab21d3cadff1 [sys.catalog]: This master's current role is: LEADER
I20260805 23:14:52.862066 15852 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27fda1fe5d744f918107ab21d3cadff1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 27fda1fe5d744f918107ab21d3cadff1. Latest consensus state: current_term: 1 leader_uuid: "27fda1fe5d744f918107ab21d3cadff1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27fda1fe5d744f918107ab21d3cadff1" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 35085 } } }
I20260805 23:14:52.862164 15852 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27fda1fe5d744f918107ab21d3cadff1 [sys.catalog]: This master's current role is: LEADER
I20260805 23:14:52.862650 15867 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:14:52.866060 15867 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:14:52.870633 15848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:52.861322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59841 (local address 127.25.92.254:35085)
0805 23:14:52.861471 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0805 23:14:52.861477 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:14:52.861735 (+ 258us) server_negotiation.cc:407] Connection header received
0805 23:14:52.861873 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:14:52.861877 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:14:52.861925 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:14:52.862001 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:14:52.863048 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:52.863587 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:52.864662 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:52.864975 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:52.865099 (+ 124us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:52.866666 (+ 1567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:14:52.866670 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:14:52.866672 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:14:52.866705 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:14:52.868287 (+ 1582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:14:52.868863 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:14:52.868866 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:14:52.868868 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:14:52.868955 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:14:52.869324 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:14:52.869327 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:14:52.869329 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:14:52.869590 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:14:52.869727 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0805 23:14:52.870376 (+ 649us) server_negotiation.cc:299] Negotiation successful
0805 23:14:52.870476 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260805 23:14:52.871948 15867 catalog_manager.cc:1347] Generated new cluster ID: bda5b38779644262a90896de65ba6fac
I20260805 23:14:52.872044 15867 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:14:52.884181 15867 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:14:52.885195 15867 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:14:52.889708 15867 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 27fda1fe5d744f918107ab21d3cadff1: Generated new TSK 0
I20260805 23:14:52.890376 15867 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260805 23:14:52.895077 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 15775
W20260805 23:14:52.901445 15759 connection.cc:535] client connection to 127.25.92.254:35085 recv error: Network error: failed to read from TLS socket (remote: 127.25.92.254:35085): Connection reset by peer (error 104)
2026-08-05T23:14:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1515 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785971406251336-25971-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 23:14:52 dist-test-slave-ss4n krb5kdc[15878](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 23:14:52 dist-test-slave-ss4n krb5kdc[15878](info): set up 2 sockets
Aug 05 23:14:52 dist-test-slave-ss4n krb5kdc[15878](info): commencing operation
krb5kdc: starting...
W20260805 23:14:53.968568 25971 mini_kdc.cc:121] Time spent starting KDC: real 1.053s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785971406251336-25971-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785971406251336-25971-0/krb5kdc/test-user.keytab.
Aug 05 23:14:54 dist-test-slave-ss4n krb5kdc[15878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971694, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T23:14:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:14:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.92.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.92.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.92.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab.
Entry for principal kudu/127.25.92.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab.
I20260805 23:14:54.110105 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785971406251336-25971-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785971406251336-25971-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785971406251336-25971-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.254:41979
--webserver_interface=127.25.92.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab
--principal=kudu/127.25.92.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:37769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.92.254:41979
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:14:54.225713 15892 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:14:54.226055 15892 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:14:54.226140 15892 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:14:54.230326 15892 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:14:54.230425 15892 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:14:54.230468 15892 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:14:54.230525 15892 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:14:54.230566 15892 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:14:54.236044 15892 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:37769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785971406251336-25971-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785971406251336-25971-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.92.254:41979
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab
--principal=kudu/127.25.92.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.254:41979
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.92.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15892
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785971406251336-25971-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:14:54.237190 15892 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:14:54.238346 15892 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:14:54.247272 15898 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:14:54.247280 15897 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:14:54.247502 15892 server_base.cc:1034] running on GCE node
W20260805 23:14:54.247272 15900 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:14:54.248198 15892 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:14:54.249365 15892 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:14:54.250605 15892 hybrid_clock.cc:648] HybridClock initialized: now 1785971694250569 us; error 40 us; skew 500 ppm
Aug 05 23:14:54 dist-test-slave-ss4n krb5kdc[15878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971694, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:14:54.253528 15892 init.cc:377] Logged in from keytab as kudu/127.25.92.254@KRBTEST.COM (short username kudu)
I20260805 23:14:54.254544 15892 webserver.cc:460] Webserver started at http://127.25.92.254:34449/ using document root <none> and password file <none>
I20260805 23:14:54.255126 15892 fs_manager.cc:359] Metadata directory not provided
I20260805 23:14:54.255221 15892 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:14:54.255453 15892 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:14:54.257601 15892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785971406251336-25971-0/minicluster-data/master-0/data/instance:
uuid: "619beb246a144e09bbbaed390a10ead9"
format_stamp: "Formatted at 2026-08-05 23:14:54 on dist-test-slave-ss4n"
server_key: "d52a993213df0efa179cda62bdc9f567"
server_key_iv: "d32c89514367f453c658851f6cb83102"
server_key_version: "encryptionkey@0"
I20260805 23:14:54.258172 15892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785971406251336-25971-0/minicluster-data/master-0/wal/instance:
uuid: "619beb246a144e09bbbaed390a10ead9"
format_stamp: "Formatted at 2026-08-05 23:14:54 on dist-test-slave-ss4n"
server_key: "d52a993213df0efa179cda62bdc9f567"
server_key_iv: "d32c89514367f453c658851f6cb83102"
server_key_version: "encryptionkey@0"
I20260805 23:14:54.261914 15892 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260805 23:14:54.264324 15907 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:14:54.265652 15892 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260805 23:14:54.265794 15892 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785971406251336-25971-0/minicluster-data/master-0/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785971406251336-25971-0/minicluster-data/master-0/wal
uuid: "619beb246a144e09bbbaed390a10ead9"
format_stamp: "Formatted at 2026-08-05 23:14:54 on dist-test-slave-ss4n"
server_key: "d52a993213df0efa179cda62bdc9f567"
server_key_iv: "d32c89514367f453c658851f6cb83102"
server_key_version: "encryptionkey@0"
I20260805 23:14:54.265949 15892 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785971406251336-25971-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785971406251336-25971-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785971406251336-25971-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:14:54.278504 15892 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:14:54.281937 15892 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:14:54.282157 15892 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:14:54.289779 15892 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.254:41979
I20260805 23:14:54.289805 15962 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.254:41979 every 8 connection(s)
I20260805 23:14:54.290735 15892 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
I20260805 23:14:54.293776 15963 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:14:54.297463 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 15892
I20260805 23:14:54.297642 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785971406251336-25971-0/minicluster-data/master-0/wal/instance
I20260805 23:14:54.297983 25971 external_mini_cluster.cc:1428] Setting key ff00b31839f524d03db6f04897e3df4d
I20260805 23:14:54.301309 15963 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 619beb246a144e09bbbaed390a10ead9: Bootstrap starting.
I20260805 23:14:54.303789 15963 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 619beb246a144e09bbbaed390a10ead9: Neither blocks nor log segments found. Creating new log.
I20260805 23:14:54.304618 15963 log.cc:824] T 00000000000000000000000000000000 P 619beb246a144e09bbbaed390a10ead9: Log is configured to *not* fsync() on all Append() calls
I20260805 23:14:54.306783 15963 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 619beb246a144e09bbbaed390a10ead9: No bootstrap required, opened a new log
I20260805 23:14:54.307659 15966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:54.299599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54457 (local address 127.25.92.254:41979)
0805 23:14:54.300097 (+ 498us) server_negotiation.cc:206] Beginning negotiation
0805 23:14:54.300137 (+ 40us) server_negotiation.cc:399] Waiting for connection header
0805 23:14:54.300173 (+ 36us) server_negotiation.cc:407] Connection header received
0805 23:14:54.300800 (+ 627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:14:54.300817 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:14:54.301167 (+ 350us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:14:54.301552 (+ 385us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:14:54.301984 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:54.302974 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:54.303885 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:54.304403 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:54.304577 (+ 174us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:54.305793 (+ 1216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:14:54.305798 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:14:54.305813 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:14:54.306243 (+ 430us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:14:54.306301 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:14:54.306326 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:14:54.306490 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0805 23:14:54.306700 (+ 210us) server_negotiation.cc:299] Negotiation successful
0805 23:14:54.306854 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":153,"threads_started":1}
I20260805 23:14:54.310825 25971 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260805 23:14:54.311251 15963 raft_consensus.cc:348] T 00000000000000000000000000000000 P 619beb246a144e09bbbaed390a10ead9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "619beb246a144e09bbbaed390a10ead9" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 41979 } }
I20260805 23:14:54.311510 15963 raft_consensus.cc:374] T 00000000000000000000000000000000 P 619beb246a144e09bbbaed390a10ead9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:14:54.311589 15963 raft_consensus.cc:729] T 00000000000000000000000000000000 P 619beb246a144e09bbbaed390a10ead9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 619beb246a144e09bbbaed390a10ead9, State: Initialized, Role: FOLLOWER
I20260805 23:14:54.312052 15963 consensus_queue.cc:260] T 00000000000000000000000000000000 P 619beb246a144e09bbbaed390a10ead9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "619beb246a144e09bbbaed390a10ead9" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 41979 } }
I20260805 23:14:54.312261 15963 raft_consensus.cc:388] T 00000000000000000000000000000000 P 619beb246a144e09bbbaed390a10ead9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:14:54.312335 15963 raft_consensus.cc:482] T 00000000000000000000000000000000 P 619beb246a144e09bbbaed390a10ead9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:14:54.312435 15963 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 619beb246a144e09bbbaed390a10ead9 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:14:54.313740 15963 raft_consensus.cc:504] T 00000000000000000000000000000000 P 619beb246a144e09bbbaed390a10ead9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "619beb246a144e09bbbaed390a10ead9" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 41979 } }
I20260805 23:14:54.314121 15963 leader_election.cc:302] T 00000000000000000000000000000000 P 619beb246a144e09bbbaed390a10ead9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 619beb246a144e09bbbaed390a10ead9; no voters:
I20260805 23:14:54.314461 15963 leader_election.cc:288] T 00000000000000000000000000000000 P 619beb246a144e09bbbaed390a10ead9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 23:14:54.314605 15973 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 619beb246a144e09bbbaed390a10ead9 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:14:54.314806 15973 raft_consensus.cc:686] T 00000000000000000000000000000000 P 619beb246a144e09bbbaed390a10ead9 [term 1 LEADER]: Becoming Leader. State: Replica: 619beb246a144e09bbbaed390a10ead9, State: Running, Role: LEADER
I20260805 23:14:54.315250 15973 consensus_queue.cc:237] T 00000000000000000000000000000000 P 619beb246a144e09bbbaed390a10ead9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "619beb246a144e09bbbaed390a10ead9" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 41979 } }
I20260805 23:14:54.315788 15963 sys_catalog.cc:564] T 00000000000000000000000000000000 P 619beb246a144e09bbbaed390a10ead9 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:14:54.316532 15973 sys_catalog.cc:455] T 00000000000000000000000000000000 P 619beb246a144e09bbbaed390a10ead9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 619beb246a144e09bbbaed390a10ead9. Latest consensus state: current_term: 1 leader_uuid: "619beb246a144e09bbbaed390a10ead9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "619beb246a144e09bbbaed390a10ead9" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 41979 } } }
I20260805 23:14:54.317102 15974 sys_catalog.cc:455] T 00000000000000000000000000000000 P 619beb246a144e09bbbaed390a10ead9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "619beb246a144e09bbbaed390a10ead9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "619beb246a144e09bbbaed390a10ead9" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 41979 } } }
I20260805 23:14:54.317224 15973 sys_catalog.cc:458] T 00000000000000000000000000000000 P 619beb246a144e09bbbaed390a10ead9 [sys.catalog]: This master's current role is: LEADER
I20260805 23:14:54.317301 15974 sys_catalog.cc:458] T 00000000000000000000000000000000 P 619beb246a144e09bbbaed390a10ead9 [sys.catalog]: This master's current role is: LEADER
I20260805 23:14:54.317754 15982 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:14:54.318416 15966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:54.313114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37303 (local address 127.25.92.254:41979)
0805 23:14:54.313256 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0805 23:14:54.313261 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:14:54.313401 (+ 140us) server_negotiation.cc:407] Connection header received
0805 23:14:54.313585 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:14:54.313588 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:14:54.313638 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:14:54.313739 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:14:54.314035 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:54.314783 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:54.315926 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:54.316280 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:54.316395 (+ 115us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:54.317464 (+ 1069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:14:54.317467 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:14:54.317470 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:14:54.317520 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:14:54.317549 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:14:54.317563 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:14:54.317673 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0805 23:14:54.318215 (+ 542us) server_negotiation.cc:299] Negotiation successful
0805 23:14:54.318260 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260805 23:14:54.321277 15982 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:14:54.327075 15982 catalog_manager.cc:1347] Generated new cluster ID: 493e4ffbbc7e4e8cacb7e14cdfde4a74
I20260805 23:14:54.327167 15982 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:14:54.336031 15982 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:14:54.337294 15982 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:14:54.350379 15982 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 619beb246a144e09bbbaed390a10ead9: Generated new TSK 0
I20260805 23:14:54.351344 15982 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260805 23:14:54.359094 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 15892
W20260805 23:14:54.365679 15876 connection.cc:535] client connection to 127.25.92.254:41979 recv error: Network error: failed to read from TLS socket (remote: 127.25.92.254:41979): Connection reset by peer (error 104)
2026-08-05T23:14:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1464 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785971406251336-25971-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 23:14:54 dist-test-slave-ss4n krb5kdc[15995](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 23:14:54 dist-test-slave-ss4n krb5kdc[15995](info): set up 2 sockets
Aug 05 23:14:54 dist-test-slave-ss4n krb5kdc[15995](info): commencing operation
krb5kdc: starting...
W20260805 23:14:55.420197 25971 mini_kdc.cc:121] Time spent starting KDC: real 1.040s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785971406251336-25971-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785971406251336-25971-0/krb5kdc/test-user.keytab.
Aug 05 23:14:55 dist-test-slave-ss4n krb5kdc[15995](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971695, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T23:14:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:14:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.92.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.92.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.92.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab.
Entry for principal kudu/127.25.92.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab.
I20260805 23:14:55.561105 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785971406251336-25971-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785971406251336-25971-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785971406251336-25971-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.254:35303
--webserver_interface=127.25.92.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab
--principal=kudu/127.25.92.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:46767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.92.254:35303
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:14:55.680068 16008 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:14:55.680444 16008 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:14:55.680540 16008 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:14:55.684832 16008 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:14:55.684958 16008 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:14:55.684998 16008 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:14:55.685039 16008 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:14:55.685074 16008 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:14:55.690717 16008 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:46767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785971406251336-25971-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785971406251336-25971-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.92.254:35303
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab
--principal=kudu/127.25.92.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.254:35303
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.92.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16008
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785971406251336-25971-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:14:55.691903 16008 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:14:55.692989 16008 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:14:55.701946 16014 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:14:55.702107 16008 server_base.cc:1034] running on GCE node
W20260805 23:14:55.701946 16013 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:14:55.702380 16016 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:14:55.703096 16008 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:14:55.704313 16008 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:14:55.705647 16008 hybrid_clock.cc:648] HybridClock initialized: now 1785971695705509 us; error 70 us; skew 500 ppm
Aug 05 23:14:55 dist-test-slave-ss4n krb5kdc[15995](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971695, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:14:55.708837 16008 init.cc:377] Logged in from keytab as kudu/127.25.92.254@KRBTEST.COM (short username kudu)
I20260805 23:14:55.709923 16008 webserver.cc:460] Webserver started at http://127.25.92.254:33067/ using document root <none> and password file <none>
I20260805 23:14:55.710482 16008 fs_manager.cc:359] Metadata directory not provided
I20260805 23:14:55.710543 16008 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:14:55.710713 16008 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:14:55.712867 16008 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785971406251336-25971-0/minicluster-data/master-0/data/instance:
uuid: "71f55be431ee43a392a00bdd102ca907"
format_stamp: "Formatted at 2026-08-05 23:14:55 on dist-test-slave-ss4n"
server_key: "78722da48089d87b4c038b80feb7011a"
server_key_iv: "d31953b8d54ea9fd2a937b687c305ec2"
server_key_version: "encryptionkey@0"
I20260805 23:14:55.713439 16008 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785971406251336-25971-0/minicluster-data/master-0/wal/instance:
uuid: "71f55be431ee43a392a00bdd102ca907"
format_stamp: "Formatted at 2026-08-05 23:14:55 on dist-test-slave-ss4n"
server_key: "78722da48089d87b4c038b80feb7011a"
server_key_iv: "d31953b8d54ea9fd2a937b687c305ec2"
server_key_version: "encryptionkey@0"
I20260805 23:14:55.717515 16008 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.000s
I20260805 23:14:55.720304 16023 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:14:55.721810 16008 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260805 23:14:55.721940 16008 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785971406251336-25971-0/minicluster-data/master-0/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785971406251336-25971-0/minicluster-data/master-0/wal
uuid: "71f55be431ee43a392a00bdd102ca907"
format_stamp: "Formatted at 2026-08-05 23:14:55 on dist-test-slave-ss4n"
server_key: "78722da48089d87b4c038b80feb7011a"
server_key_iv: "d31953b8d54ea9fd2a937b687c305ec2"
server_key_version: "encryptionkey@0"
I20260805 23:14:55.722052 16008 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785971406251336-25971-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785971406251336-25971-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785971406251336-25971-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:14:55.727516 16008 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:14:55.730942 16008 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:14:55.731115 16008 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:14:55.739384 16008 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.254:35303
I20260805 23:14:55.739398 16076 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.254:35303 every 8 connection(s)
I20260805 23:14:55.740445 16008 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
I20260805 23:14:55.743299 16077 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:14:55.748498 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 16008
I20260805 23:14:55.748638 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785971406251336-25971-0/minicluster-data/master-0/wal/instance
I20260805 23:14:55.748955 25971 external_mini_cluster.cc:1428] Setting key 5258078eaaa3f2516629a1aad49d2b30
I20260805 23:14:55.749161 16077 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 71f55be431ee43a392a00bdd102ca907: Bootstrap starting.
I20260805 23:14:55.751662 16077 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 71f55be431ee43a392a00bdd102ca907: Neither blocks nor log segments found. Creating new log.
I20260805 23:14:55.752557 16077 log.cc:824] T 00000000000000000000000000000000 P 71f55be431ee43a392a00bdd102ca907: Log is configured to *not* fsync() on all Append() calls
I20260805 23:14:55.754518 16080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:55.750439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55003 (local address 127.25.92.254:35303)
0805 23:14:55.750891 (+ 452us) server_negotiation.cc:206] Beginning negotiation
0805 23:14:55.750907 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0805 23:14:55.750943 (+ 36us) server_negotiation.cc:407] Connection header received
0805 23:14:55.751637 (+ 694us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:14:55.751655 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:14:55.752003 (+ 348us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:14:55.752316 (+ 313us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:14:55.753262 (+ 946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:14:55.753275 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:14:55.753287 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:14:55.753298 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:14:55.753353 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:14:55.753373 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:14:55.753491 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0805 23:14:55.753631 (+ 140us) server_negotiation.cc:299] Negotiation successful
0805 23:14:55.753798 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":141,"threads_started":1}
I20260805 23:14:55.754956 16077 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 71f55be431ee43a392a00bdd102ca907: No bootstrap required, opened a new log
I20260805 23:14:55.757676 25971 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260805 23:14:55.759620 16077 raft_consensus.cc:348] T 00000000000000000000000000000000 P 71f55be431ee43a392a00bdd102ca907 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71f55be431ee43a392a00bdd102ca907" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 35303 } }
I20260805 23:14:55.760061 16077 raft_consensus.cc:374] T 00000000000000000000000000000000 P 71f55be431ee43a392a00bdd102ca907 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:14:55.760162 16077 raft_consensus.cc:729] T 00000000000000000000000000000000 P 71f55be431ee43a392a00bdd102ca907 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71f55be431ee43a392a00bdd102ca907, State: Initialized, Role: FOLLOWER
I20260805 23:14:55.760710 16077 consensus_queue.cc:260] T 00000000000000000000000000000000 P 71f55be431ee43a392a00bdd102ca907 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71f55be431ee43a392a00bdd102ca907" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 35303 } }
I20260805 23:14:55.760921 16077 raft_consensus.cc:388] T 00000000000000000000000000000000 P 71f55be431ee43a392a00bdd102ca907 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:14:55.761006 16077 raft_consensus.cc:482] T 00000000000000000000000000000000 P 71f55be431ee43a392a00bdd102ca907 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:14:55.761147 16077 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 71f55be431ee43a392a00bdd102ca907 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:14:55.761739 16080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:55.759897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35291 (local address 127.25.92.254:35303)
0805 23:14:55.760033 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0805 23:14:55.760037 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:14:55.760207 (+ 170us) server_negotiation.cc:407] Connection header received
0805 23:14:55.760332 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:14:55.760336 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:14:55.760377 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:14:55.760478 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:14:55.761311 (+ 833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:14:55.761314 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:14:55.761317 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:14:55.761335 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:14:55.761361 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:14:55.761372 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:14:55.761467 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0805 23:14:55.761562 (+ 95us) server_negotiation.cc:299] Negotiation successful
0805 23:14:55.761605 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260805 23:14:55.762575 16077 raft_consensus.cc:504] T 00000000000000000000000000000000 P 71f55be431ee43a392a00bdd102ca907 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71f55be431ee43a392a00bdd102ca907" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 35303 } }
I20260805 23:14:55.762986 16077 leader_election.cc:302] T 00000000000000000000000000000000 P 71f55be431ee43a392a00bdd102ca907 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 71f55be431ee43a392a00bdd102ca907; no voters:
I20260805 23:14:55.763325 16077 leader_election.cc:288] T 00000000000000000000000000000000 P 71f55be431ee43a392a00bdd102ca907 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 23:14:55.763454 16088 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 71f55be431ee43a392a00bdd102ca907 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:14:55.763700 16088 raft_consensus.cc:686] T 00000000000000000000000000000000 P 71f55be431ee43a392a00bdd102ca907 [term 1 LEADER]: Becoming Leader. State: Replica: 71f55be431ee43a392a00bdd102ca907, State: Running, Role: LEADER
I20260805 23:14:55.764077 16088 consensus_queue.cc:237] T 00000000000000000000000000000000 P 71f55be431ee43a392a00bdd102ca907 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71f55be431ee43a392a00bdd102ca907" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 35303 } }
I20260805 23:14:55.764700 16077 sys_catalog.cc:564] T 00000000000000000000000000000000 P 71f55be431ee43a392a00bdd102ca907 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:14:55.765785 16089 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71f55be431ee43a392a00bdd102ca907 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "71f55be431ee43a392a00bdd102ca907" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71f55be431ee43a392a00bdd102ca907" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 35303 } } }
I20260805 23:14:55.765918 16089 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71f55be431ee43a392a00bdd102ca907 [sys.catalog]: This master's current role is: LEADER
I20260805 23:14:55.766304 16097 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:14:55.766300 16090 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71f55be431ee43a392a00bdd102ca907 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 71f55be431ee43a392a00bdd102ca907. Latest consensus state: current_term: 1 leader_uuid: "71f55be431ee43a392a00bdd102ca907" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71f55be431ee43a392a00bdd102ca907" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 35303 } } }
I20260805 23:14:55.766484 16090 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71f55be431ee43a392a00bdd102ca907 [sys.catalog]: This master's current role is: LEADER
I20260805 23:14:55.769445 16097 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:14:55.775040 16097 catalog_manager.cc:1347] Generated new cluster ID: dc1b09ca94804e6baff8d7f3f806172a
I20260805 23:14:55.775120 16097 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:14:55.793140 16097 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:14:55.793542 16097 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:14:55.801369 16097 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 71f55be431ee43a392a00bdd102ca907: Generated new TSK 0
I20260805 23:14:55.802229 16097 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260805 23:14:55.805943 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 16008
2026-08-05T23:14:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1446 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785971406251336-25971-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 23:14:55 dist-test-slave-ss4n krb5kdc[16109](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 23:14:55 dist-test-slave-ss4n krb5kdc[16109](info): set up 2 sockets
Aug 05 23:14:55 dist-test-slave-ss4n krb5kdc[16109](info): commencing operation
krb5kdc: starting...
W20260805 23:14:57.108198 25971 mini_kdc.cc:121] Time spent starting KDC: real 1.281s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785971406251336-25971-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785971406251336-25971-0/krb5kdc/test-user.keytab.
Aug 05 23:14:57 dist-test-slave-ss4n krb5kdc[16109](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971697, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T23:14:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:14:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.92.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.92.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.92.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab.
Entry for principal kudu/127.25.92.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab.
I20260805 23:14:57.241343 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785971406251336-25971-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785971406251336-25971-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785971406251336-25971-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.254:34263
--webserver_interface=127.25.92.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab
--principal=kudu/127.25.92.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:44667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.92.254:34263
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:14:57.374208 16122 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:14:57.374502 16122 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:14:57.374547 16122 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:14:57.378750 16122 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:14:57.378823 16122 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:14:57.378844 16122 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:14:57.378861 16122 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:14:57.378877 16122 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:14:57.384851 16122 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:44667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785971406251336-25971-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785971406251336-25971-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.92.254:34263
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab
--principal=kudu/127.25.92.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.254:34263
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.92.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16122
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785971406251336-25971-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:14:57.386116 16122 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:14:57.387100 16122 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:14:57.395794 16127 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:14:57.395788 16128 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:14:57.395788 16130 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:14:57.396234 16122 server_base.cc:1034] running on GCE node
I20260805 23:14:57.396875 16122 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:14:57.398131 16122 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:14:57.399343 16122 hybrid_clock.cc:648] HybridClock initialized: now 1785971697399314 us; error 40 us; skew 500 ppm
Aug 05 23:14:57 dist-test-slave-ss4n krb5kdc[16109](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971697, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:14:57.402346 16122 init.cc:377] Logged in from keytab as kudu/127.25.92.254@KRBTEST.COM (short username kudu)
I20260805 23:14:57.403259 16122 webserver.cc:460] Webserver started at http://127.25.92.254:46125/ using document root <none> and password file <none>
I20260805 23:14:57.403784 16122 fs_manager.cc:359] Metadata directory not provided
I20260805 23:14:57.403843 16122 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:14:57.404008 16122 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:14:57.406188 16122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785971406251336-25971-0/minicluster-data/master-0/data/instance:
uuid: "11594046bd2a4611ba873f704e67e1bd"
format_stamp: "Formatted at 2026-08-05 23:14:57 on dist-test-slave-ss4n"
server_key: "5317ee825606e3ebd807eb1c3127b072"
server_key_iv: "cc362dfdbc3aaa358917478581e130b3"
server_key_version: "encryptionkey@0"
I20260805 23:14:57.406737 16122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785971406251336-25971-0/minicluster-data/master-0/wal/instance:
uuid: "11594046bd2a4611ba873f704e67e1bd"
format_stamp: "Formatted at 2026-08-05 23:14:57 on dist-test-slave-ss4n"
server_key: "5317ee825606e3ebd807eb1c3127b072"
server_key_iv: "cc362dfdbc3aaa358917478581e130b3"
server_key_version: "encryptionkey@0"
I20260805 23:14:57.410823 16122 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260805 23:14:57.413373 16138 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:14:57.414783 16122 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:14:57.414951 16122 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785971406251336-25971-0/minicluster-data/master-0/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785971406251336-25971-0/minicluster-data/master-0/wal
uuid: "11594046bd2a4611ba873f704e67e1bd"
format_stamp: "Formatted at 2026-08-05 23:14:57 on dist-test-slave-ss4n"
server_key: "5317ee825606e3ebd807eb1c3127b072"
server_key_iv: "cc362dfdbc3aaa358917478581e130b3"
server_key_version: "encryptionkey@0"
I20260805 23:14:57.415089 16122 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785971406251336-25971-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785971406251336-25971-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785971406251336-25971-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:14:57.430331 16122 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:14:57.433574 16122 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:14:57.433808 16122 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:14:57.441293 16122 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.254:34263
I20260805 23:14:57.441320 16191 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.254:34263 every 8 connection(s)
I20260805 23:14:57.442447 16122 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
I20260805 23:14:57.445274 16192 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:14:57.448894 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 16122
I20260805 23:14:57.449043 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785971406251336-25971-0/minicluster-data/master-0/wal/instance
I20260805 23:14:57.449383 25971 external_mini_cluster.cc:1428] Setting key 793dc4a87c2cc9c1f22dc1361b0d9a58
I20260805 23:14:57.452414 16192 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11594046bd2a4611ba873f704e67e1bd: Bootstrap starting.
I20260805 23:14:57.454756 16192 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 11594046bd2a4611ba873f704e67e1bd: Neither blocks nor log segments found. Creating new log.
I20260805 23:14:57.455559 16192 log.cc:824] T 00000000000000000000000000000000 P 11594046bd2a4611ba873f704e67e1bd: Log is configured to *not* fsync() on all Append() calls
I20260805 23:14:57.457813 16192 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11594046bd2a4611ba873f704e67e1bd: No bootstrap required, opened a new log
Aug 05 23:14:57 dist-test-slave-ss4n krb5kdc[16109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971697, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM
I20260805 23:14:57.461580 16192 raft_consensus.cc:348] T 00000000000000000000000000000000 P 11594046bd2a4611ba873f704e67e1bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11594046bd2a4611ba873f704e67e1bd" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 34263 } }
I20260805 23:14:57.461874 16192 raft_consensus.cc:374] T 00000000000000000000000000000000 P 11594046bd2a4611ba873f704e67e1bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:14:57.461956 16192 raft_consensus.cc:729] T 00000000000000000000000000000000 P 11594046bd2a4611ba873f704e67e1bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11594046bd2a4611ba873f704e67e1bd, State: Initialized, Role: FOLLOWER
I20260805 23:14:57.462478 16192 consensus_queue.cc:260] T 00000000000000000000000000000000 P 11594046bd2a4611ba873f704e67e1bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11594046bd2a4611ba873f704e67e1bd" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 34263 } }
I20260805 23:14:57.462641 16192 raft_consensus.cc:388] T 00000000000000000000000000000000 P 11594046bd2a4611ba873f704e67e1bd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:14:57.462728 16192 raft_consensus.cc:482] T 00000000000000000000000000000000 P 11594046bd2a4611ba873f704e67e1bd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:14:57.462828 16192 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 11594046bd2a4611ba873f704e67e1bd [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:14:57.463804 16192 raft_consensus.cc:504] T 00000000000000000000000000000000 P 11594046bd2a4611ba873f704e67e1bd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11594046bd2a4611ba873f704e67e1bd" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 34263 } }
I20260805 23:14:57.464175 16192 leader_election.cc:302] T 00000000000000000000000000000000 P 11594046bd2a4611ba873f704e67e1bd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 11594046bd2a4611ba873f704e67e1bd; no voters:
I20260805 23:14:57.464543 16192 leader_election.cc:288] T 00000000000000000000000000000000 P 11594046bd2a4611ba873f704e67e1bd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 23:14:57.464665 16197 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 11594046bd2a4611ba873f704e67e1bd [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:14:57.464890 16197 raft_consensus.cc:686] T 00000000000000000000000000000000 P 11594046bd2a4611ba873f704e67e1bd [term 1 LEADER]: Becoming Leader. State: Replica: 11594046bd2a4611ba873f704e67e1bd, State: Running, Role: LEADER
I20260805 23:14:57.465296 16197 consensus_queue.cc:237] T 00000000000000000000000000000000 P 11594046bd2a4611ba873f704e67e1bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11594046bd2a4611ba873f704e67e1bd" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 34263 } }
I20260805 23:14:57.465538 16195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:57.451043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39819 (local address 127.25.92.254:34263)
0805 23:14:57.451592 (+ 549us) server_negotiation.cc:206] Beginning negotiation
0805 23:14:57.451609 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0805 23:14:57.451648 (+ 39us) server_negotiation.cc:407] Connection header received
0805 23:14:57.452293 (+ 645us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:14:57.452310 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:14:57.452724 (+ 414us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:14:57.453074 (+ 350us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:14:57.453934 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:57.455007 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:57.455900 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:57.456389 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:57.456597 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:57.459430 (+ 2833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:14:57.459435 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:14:57.459450 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:14:57.459479 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:14:57.462160 (+ 2681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:14:57.462668 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:14:57.462675 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:14:57.462680 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:14:57.462851 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:14:57.463148 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:14:57.463152 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:14:57.463153 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:14:57.463567 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:14:57.463786 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0805 23:14:57.464091 (+ 305us) server_negotiation.cc:299] Negotiation successful
0805 23:14:57.464429 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":170,"threads_started":1}
I20260805 23:14:57.465956 16192 sys_catalog.cc:564] T 00000000000000000000000000000000 P 11594046bd2a4611ba873f704e67e1bd [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:14:57.467440 16199 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11594046bd2a4611ba873f704e67e1bd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 11594046bd2a4611ba873f704e67e1bd. Latest consensus state: current_term: 1 leader_uuid: "11594046bd2a4611ba873f704e67e1bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11594046bd2a4611ba873f704e67e1bd" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 34263 } } }
I20260805 23:14:57.467566 16199 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11594046bd2a4611ba873f704e67e1bd [sys.catalog]: This master's current role is: LEADER
I20260805 23:14:57.467935 16198 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11594046bd2a4611ba873f704e67e1bd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "11594046bd2a4611ba873f704e67e1bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11594046bd2a4611ba873f704e67e1bd" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 34263 } } }
I20260805 23:14:57.468079 16198 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11594046bd2a4611ba873f704e67e1bd [sys.catalog]: This master's current role is: LEADER
I20260805 23:14:57.468631 16206 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:14:57.469316 25971 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260805 23:14:57.472286 16206 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:14:57.479198 16206 catalog_manager.cc:1347] Generated new cluster ID: 4e8ea18a84b04b0885f01d8389300bfc
I20260805 23:14:57.479271 16206 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:14:57.481329 16195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:57.471617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52949 (local address 127.25.92.254:34263)
0805 23:14:57.471719 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0805 23:14:57.471724 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:14:57.471972 (+ 248us) server_negotiation.cc:407] Connection header received
0805 23:14:57.472169 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:14:57.472172 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:14:57.472222 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:14:57.472277 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:14:57.473270 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:57.474146 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:57.475358 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:57.475700 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:57.475817 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:57.477376 (+ 1559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:14:57.477380 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:14:57.477385 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:14:57.477414 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:14:57.479356 (+ 1942us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:14:57.479901 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:14:57.479904 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:14:57.479906 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:14:57.479997 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:14:57.480386 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:14:57.480390 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:14:57.480392 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:14:57.480644 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:14:57.480783 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0805 23:14:57.481019 (+ 236us) server_negotiation.cc:299] Negotiation successful
0805 23:14:57.481152 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260805 23:14:57.524118 16206 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:14:57.525718 16206 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:14:57.530819 16206 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 11594046bd2a4611ba873f704e67e1bd: Generated new TSK 0
I20260805 23:14:57.531661 16206 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260805 23:14:57.540184 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 16122
2026-08-05T23:14:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1734 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785971406251336-25971-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 23:14:57 dist-test-slave-ss4n krb5kdc[16223](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 23:14:57 dist-test-slave-ss4n krb5kdc[16223](info): set up 2 sockets
Aug 05 23:14:57 dist-test-slave-ss4n krb5kdc[16223](info): commencing operation
krb5kdc: starting...
W20260805 23:14:58.605866 25971 mini_kdc.cc:121] Time spent starting KDC: real 1.045s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785971406251336-25971-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785971406251336-25971-0/krb5kdc/test-user.keytab.
Aug 05 23:14:58 dist-test-slave-ss4n krb5kdc[16223](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971698, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T23:14:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:14:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.92.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.92.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.92.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab.
Entry for principal kudu/127.25.92.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab.
I20260805 23:14:58.748848 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785971406251336-25971-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785971406251336-25971-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785971406251336-25971-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.254:40015
--webserver_interface=127.25.92.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab
--principal=kudu/127.25.92.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:41303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.92.254:40015
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:14:58.861392 16236 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:14:58.861725 16236 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:14:58.861771 16236 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:14:58.865880 16236 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:14:58.865947 16236 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:14:58.865965 16236 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:14:58.865983 16236 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:14:58.865998 16236 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:14:58.871361 16236 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:41303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785971406251336-25971-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785971406251336-25971-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.92.254:40015
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab
--principal=kudu/127.25.92.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.254:40015
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.92.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16236
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785971406251336-25971-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:14:58.872416 16236 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:14:58.873407 16236 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:14:58.882576 16241 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:14:58.882790 16236 server_base.cc:1034] running on GCE node
W20260805 23:14:58.882643 16242 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:14:58.882681 16244 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:14:58.883472 16236 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:14:58.884656 16236 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:14:58.885849 16236 hybrid_clock.cc:648] HybridClock initialized: now 1785971698885828 us; error 34 us; skew 500 ppm
Aug 05 23:14:58 dist-test-slave-ss4n krb5kdc[16223](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971698, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:14:58.893163 16236 init.cc:377] Logged in from keytab as kudu/127.25.92.254@KRBTEST.COM (short username kudu)
I20260805 23:14:58.894227 16236 webserver.cc:460] Webserver started at http://127.25.92.254:46455/ using document root <none> and password file <none>
I20260805 23:14:58.894778 16236 fs_manager.cc:359] Metadata directory not provided
I20260805 23:14:58.894837 16236 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:14:58.895102 16236 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:14:58.897162 16236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785971406251336-25971-0/minicluster-data/master-0/data/instance:
uuid: "b9f0b2c07e2c43a6805a91b93f376f10"
format_stamp: "Formatted at 2026-08-05 23:14:58 on dist-test-slave-ss4n"
server_key: "4129ab394850161e70b8f98b17b21d2e"
server_key_iv: "fee3da429561fa672e996dbc8fedf82b"
server_key_version: "encryptionkey@0"
I20260805 23:14:58.897763 16236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785971406251336-25971-0/minicluster-data/master-0/wal/instance:
uuid: "b9f0b2c07e2c43a6805a91b93f376f10"
format_stamp: "Formatted at 2026-08-05 23:14:58 on dist-test-slave-ss4n"
server_key: "4129ab394850161e70b8f98b17b21d2e"
server_key_iv: "fee3da429561fa672e996dbc8fedf82b"
server_key_version: "encryptionkey@0"
I20260805 23:14:58.901436 16236 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260805 23:14:58.903945 16251 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:14:58.905354 16236 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260805 23:14:58.905473 16236 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785971406251336-25971-0/minicluster-data/master-0/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785971406251336-25971-0/minicluster-data/master-0/wal
uuid: "b9f0b2c07e2c43a6805a91b93f376f10"
format_stamp: "Formatted at 2026-08-05 23:14:58 on dist-test-slave-ss4n"
server_key: "4129ab394850161e70b8f98b17b21d2e"
server_key_iv: "fee3da429561fa672e996dbc8fedf82b"
server_key_version: "encryptionkey@0"
I20260805 23:14:58.905645 16236 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785971406251336-25971-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785971406251336-25971-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785971406251336-25971-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:14:58.926113 16236 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:14:58.929895 16236 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:14:58.930092 16236 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:14:58.938470 16236 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.254:40015
I20260805 23:14:58.938594 16304 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.254:40015 every 8 connection(s)
I20260805 23:14:58.939513 16236 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
I20260805 23:14:58.942662 16306 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:14:58.946190 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 16236
I20260805 23:14:58.946324 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785971406251336-25971-0/minicluster-data/master-0/wal/instance
I20260805 23:14:58.946617 25971 external_mini_cluster.cc:1428] Setting key 6b038113627a3c345a92d3a13d983704
I20260805 23:14:58.949657 16306 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9f0b2c07e2c43a6805a91b93f376f10: Bootstrap starting.
I20260805 23:14:58.952735 16306 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b9f0b2c07e2c43a6805a91b93f376f10: Neither blocks nor log segments found. Creating new log.
I20260805 23:14:58.953840 16306 log.cc:824] T 00000000000000000000000000000000 P b9f0b2c07e2c43a6805a91b93f376f10: Log is configured to *not* fsync() on all Append() calls
I20260805 23:14:58.956565 16306 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9f0b2c07e2c43a6805a91b93f376f10: No bootstrap required, opened a new log
I20260805 23:14:58.956914 16309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:58.948033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60285 (local address 127.25.92.254:40015)
0805 23:14:58.948469 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0805 23:14:58.948485 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0805 23:14:58.948522 (+ 37us) server_negotiation.cc:407] Connection header received
0805 23:14:58.949171 (+ 649us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:14:58.949189 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:14:58.949619 (+ 430us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:14:58.949970 (+ 351us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:14:58.950386 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:58.951409 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:58.952927 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:58.953571 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:58.953719 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:58.954699 (+ 980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:14:58.954707 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:14:58.954726 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:14:58.955248 (+ 522us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:14:58.955311 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:14:58.955338 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:14:58.955426 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0805 23:14:58.955716 (+ 290us) server_negotiation.cc:299] Negotiation successful
0805 23:14:58.956012 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285,"thread_start_us":144,"threads_started":1}
I20260805 23:14:58.960436 25971 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260805 23:14:58.961596 16306 raft_consensus.cc:348] T 00000000000000000000000000000000 P b9f0b2c07e2c43a6805a91b93f376f10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9f0b2c07e2c43a6805a91b93f376f10" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 40015 } }
I20260805 23:14:58.961946 16306 raft_consensus.cc:374] T 00000000000000000000000000000000 P b9f0b2c07e2c43a6805a91b93f376f10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:14:58.962011 16306 raft_consensus.cc:729] T 00000000000000000000000000000000 P b9f0b2c07e2c43a6805a91b93f376f10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9f0b2c07e2c43a6805a91b93f376f10, State: Initialized, Role: FOLLOWER
I20260805 23:14:58.962697 16306 consensus_queue.cc:260] T 00000000000000000000000000000000 P b9f0b2c07e2c43a6805a91b93f376f10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9f0b2c07e2c43a6805a91b93f376f10" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 40015 } }
I20260805 23:14:58.962927 16306 raft_consensus.cc:388] T 00000000000000000000000000000000 P b9f0b2c07e2c43a6805a91b93f376f10 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:14:58.963011 16306 raft_consensus.cc:482] T 00000000000000000000000000000000 P b9f0b2c07e2c43a6805a91b93f376f10 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:14:58.963160 16306 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b9f0b2c07e2c43a6805a91b93f376f10 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:14:58.964612 16306 raft_consensus.cc:504] T 00000000000000000000000000000000 P b9f0b2c07e2c43a6805a91b93f376f10 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9f0b2c07e2c43a6805a91b93f376f10" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 40015 } }
I20260805 23:14:58.965032 16306 leader_election.cc:302] T 00000000000000000000000000000000 P b9f0b2c07e2c43a6805a91b93f376f10 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b9f0b2c07e2c43a6805a91b93f376f10; no voters:
I20260805 23:14:58.965417 16306 leader_election.cc:288] T 00000000000000000000000000000000 P b9f0b2c07e2c43a6805a91b93f376f10 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 23:14:58.965583 16316 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b9f0b2c07e2c43a6805a91b93f376f10 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:14:58.965900 16316 raft_consensus.cc:686] T 00000000000000000000000000000000 P b9f0b2c07e2c43a6805a91b93f376f10 [term 1 LEADER]: Becoming Leader. State: Replica: b9f0b2c07e2c43a6805a91b93f376f10, State: Running, Role: LEADER
I20260805 23:14:58.966291 16316 consensus_queue.cc:237] T 00000000000000000000000000000000 P b9f0b2c07e2c43a6805a91b93f376f10 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9f0b2c07e2c43a6805a91b93f376f10" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 40015 } }
I20260805 23:14:58.966996 16306 sys_catalog.cc:564] T 00000000000000000000000000000000 P b9f0b2c07e2c43a6805a91b93f376f10 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:14:58.968274 16318 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9f0b2c07e2c43a6805a91b93f376f10 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b9f0b2c07e2c43a6805a91b93f376f10" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9f0b2c07e2c43a6805a91b93f376f10" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 40015 } } }
I20260805 23:14:58.968274 16316 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9f0b2c07e2c43a6805a91b93f376f10 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b9f0b2c07e2c43a6805a91b93f376f10. Latest consensus state: current_term: 1 leader_uuid: "b9f0b2c07e2c43a6805a91b93f376f10" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9f0b2c07e2c43a6805a91b93f376f10" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 40015 } } }
I20260805 23:14:58.968591 16316 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9f0b2c07e2c43a6805a91b93f376f10 [sys.catalog]: This master's current role is: LEADER
I20260805 23:14:58.968623 16318 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9f0b2c07e2c43a6805a91b93f376f10 [sys.catalog]: This master's current role is: LEADER
I20260805 23:14:58.969229 16309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:14:58.963268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56617 (local address 127.25.92.254:40015)
0805 23:14:58.963412 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0805 23:14:58.963417 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:14:58.963669 (+ 252us) server_negotiation.cc:407] Connection header received
0805 23:14:58.963817 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:14:58.963820 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:14:58.963868 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:14:58.963934 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:14:58.964283 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:58.964928 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:58.966208 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:14:58.966559 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:14:58.966647 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:14:58.967909 (+ 1262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:14:58.967913 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:14:58.967917 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:14:58.967932 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:14:58.967969 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:14:58.967998 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:14:58.968099 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0805 23:14:58.968979 (+ 880us) server_negotiation.cc:299] Negotiation successful
0805 23:14:58.969056 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260805 23:14:58.969606 16326 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:14:58.973161 16326 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:14:58.979629 16326 catalog_manager.cc:1347] Generated new cluster ID: 93545d13838c4bb8a41727db041da127
I20260805 23:14:58.979733 16326 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:14:58.991756 16326 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:14:58.992784 16326 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:14:58.997469 16326 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b9f0b2c07e2c43a6805a91b93f376f10: Generated new TSK 0
I20260805 23:14:58.998407 16326 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260805 23:14:59.011940 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 16236
2026-08-05T23:14:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1472 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785971406251336-25971-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 23:14:59 dist-test-slave-ss4n krb5kdc[16339](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 23:14:59 dist-test-slave-ss4n krb5kdc[16339](info): set up 2 sockets
Aug 05 23:14:59 dist-test-slave-ss4n krb5kdc[16339](info): commencing operation
krb5kdc: starting...
W20260805 23:15:00.100196 25971 mini_kdc.cc:121] Time spent starting KDC: real 1.066s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785971406251336-25971-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785971406251336-25971-0/krb5kdc/test-user.keytab.
Aug 05 23:15:00 dist-test-slave-ss4n krb5kdc[16339](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971700, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T23:15:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:15:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.92.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.92.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.92.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab.
Entry for principal kudu/127.25.92.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab.
I20260805 23:15:00.237147 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785971406251336-25971-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785971406251336-25971-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785971406251336-25971-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.254:35131
--webserver_interface=127.25.92.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab
--principal=kudu/127.25.92.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:41441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.92.254:35131
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:15:00.352315 16352 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:15:00.352722 16352 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:15:00.352818 16352 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:15:00.357117 16352 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:15:00.357249 16352 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:15:00.357288 16352 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:15:00.357339 16352 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:15:00.357414 16352 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:15:00.363224 16352 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:41441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785971406251336-25971-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785971406251336-25971-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.92.254:35131
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab
--principal=kudu/127.25.92.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.254:35131
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.92.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16352
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785971406251336-25971-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:15:00.364475 16352 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:15:00.365624 16352 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:15:00.374732 16357 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:00.374747 16360 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:00.374850 16352 server_base.cc:1034] running on GCE node
W20260805 23:15:00.374727 16358 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:00.375727 16352 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:15:00.376921 16352 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:15:00.378139 16352 hybrid_clock.cc:648] HybridClock initialized: now 1785971700378120 us; error 34 us; skew 500 ppm
Aug 05 23:15:00 dist-test-slave-ss4n krb5kdc[16339](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971700, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:15:00.381145 16352 init.cc:377] Logged in from keytab as kudu/127.25.92.254@KRBTEST.COM (short username kudu)
I20260805 23:15:00.382154 16352 webserver.cc:460] Webserver started at http://127.25.92.254:46333/ using document root <none> and password file <none>
I20260805 23:15:00.382818 16352 fs_manager.cc:359] Metadata directory not provided
I20260805 23:15:00.382920 16352 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:15:00.383170 16352 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:15:00.385368 16352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785971406251336-25971-0/minicluster-data/master-0/data/instance:
uuid: "984d87a7c9e6457288a6bac46d18f595"
format_stamp: "Formatted at 2026-08-05 23:15:00 on dist-test-slave-ss4n"
server_key: "df6410f1463f9d3117f432e3d704799e"
server_key_iv: "bab7c7cc042a5dedbfd54c1c2362cd87"
server_key_version: "encryptionkey@0"
I20260805 23:15:00.386049 16352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785971406251336-25971-0/minicluster-data/master-0/wal/instance:
uuid: "984d87a7c9e6457288a6bac46d18f595"
format_stamp: "Formatted at 2026-08-05 23:15:00 on dist-test-slave-ss4n"
server_key: "df6410f1463f9d3117f432e3d704799e"
server_key_iv: "bab7c7cc042a5dedbfd54c1c2362cd87"
server_key_version: "encryptionkey@0"
I20260805 23:15:00.390132 16352 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260805 23:15:00.392840 16367 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:00.394430 16352 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:15:00.394606 16352 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785971406251336-25971-0/minicluster-data/master-0/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785971406251336-25971-0/minicluster-data/master-0/wal
uuid: "984d87a7c9e6457288a6bac46d18f595"
format_stamp: "Formatted at 2026-08-05 23:15:00 on dist-test-slave-ss4n"
server_key: "df6410f1463f9d3117f432e3d704799e"
server_key_iv: "bab7c7cc042a5dedbfd54c1c2362cd87"
server_key_version: "encryptionkey@0"
I20260805 23:15:00.394748 16352 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785971406251336-25971-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785971406251336-25971-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785971406251336-25971-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:15:00.400082 16352 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:15:00.403823 16352 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:15:00.404060 16352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:15:00.411492 16352 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.254:35131
I20260805 23:15:00.411582 16420 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.254:35131 every 8 connection(s)
I20260805 23:15:00.412494 16352 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
I20260805 23:15:00.414203 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 16352
I20260805 23:15:00.414359 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785971406251336-25971-0/minicluster-data/master-0/wal/instance
I20260805 23:15:00.414661 25971 external_mini_cluster.cc:1428] Setting key f54e3adb6c15b71b3dde18c9fd2e53b4
I20260805 23:15:00.415709 16421 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:15:00.420539 16424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:00.416082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42217 (local address 127.25.92.254:35131)
0805 23:15:00.416526 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:00.416546 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:00.416589 (+ 43us) server_negotiation.cc:407] Connection header received
0805 23:15:00.417468 (+ 879us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:00.417522 (+ 54us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:00.417872 (+ 350us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:00.418259 (+ 387us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:00.419066 (+ 807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:00.419072 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:00.419087 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:15:00.419099 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:00.419166 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:15:00.419194 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:00.419346 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:00.419460 (+ 114us) server_negotiation.cc:299] Negotiation successful
0805 23:15:00.419624 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":107,"threads_started":1}
I20260805 23:15:00.422966 16421 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 984d87a7c9e6457288a6bac46d18f595: Bootstrap starting.
I20260805 23:15:00.423882 25971 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260805 23:15:00.425868 16421 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 984d87a7c9e6457288a6bac46d18f595: Neither blocks nor log segments found. Creating new log.
I20260805 23:15:00.426851 16421 log.cc:824] T 00000000000000000000000000000000 P 984d87a7c9e6457288a6bac46d18f595: Log is configured to *not* fsync() on all Append() calls
I20260805 23:15:00.428381 16424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:00.426316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53239 (local address 127.25.92.254:35131)
0805 23:15:00.426457 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:00.426462 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:00.426741 (+ 279us) server_negotiation.cc:407] Connection header received
0805 23:15:00.426868 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:00.426870 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:00.426906 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:00.427003 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:00.427822 (+ 819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:00.427825 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:00.427827 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:15:00.427837 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:00.427862 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:15:00.427874 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:00.427943 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:00.428130 (+ 187us) server_negotiation.cc:299] Negotiation successful
0805 23:15:00.428208 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260805 23:15:00.429451 16421 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 984d87a7c9e6457288a6bac46d18f595: No bootstrap required, opened a new log
I20260805 23:15:00.433220 16421 raft_consensus.cc:348] T 00000000000000000000000000000000 P 984d87a7c9e6457288a6bac46d18f595 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "984d87a7c9e6457288a6bac46d18f595" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 35131 } }
I20260805 23:15:00.433557 16421 raft_consensus.cc:374] T 00000000000000000000000000000000 P 984d87a7c9e6457288a6bac46d18f595 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:15:00.433638 16421 raft_consensus.cc:729] T 00000000000000000000000000000000 P 984d87a7c9e6457288a6bac46d18f595 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 984d87a7c9e6457288a6bac46d18f595, State: Initialized, Role: FOLLOWER
I20260805 23:15:00.434106 16421 consensus_queue.cc:260] T 00000000000000000000000000000000 P 984d87a7c9e6457288a6bac46d18f595 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "984d87a7c9e6457288a6bac46d18f595" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 35131 } }
I20260805 23:15:00.434269 16421 raft_consensus.cc:388] T 00000000000000000000000000000000 P 984d87a7c9e6457288a6bac46d18f595 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:15:00.434345 16421 raft_consensus.cc:482] T 00000000000000000000000000000000 P 984d87a7c9e6457288a6bac46d18f595 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:15:00.434453 16421 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 984d87a7c9e6457288a6bac46d18f595 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:15:00.435648 16421 raft_consensus.cc:504] T 00000000000000000000000000000000 P 984d87a7c9e6457288a6bac46d18f595 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "984d87a7c9e6457288a6bac46d18f595" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 35131 } }
I20260805 23:15:00.436043 16421 leader_election.cc:302] T 00000000000000000000000000000000 P 984d87a7c9e6457288a6bac46d18f595 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 984d87a7c9e6457288a6bac46d18f595; no voters:
I20260805 23:15:00.436393 16421 leader_election.cc:288] T 00000000000000000000000000000000 P 984d87a7c9e6457288a6bac46d18f595 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 23:15:00.436666 16431 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 984d87a7c9e6457288a6bac46d18f595 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:15:00.436930 16431 raft_consensus.cc:686] T 00000000000000000000000000000000 P 984d87a7c9e6457288a6bac46d18f595 [term 1 LEADER]: Becoming Leader. State: Replica: 984d87a7c9e6457288a6bac46d18f595, State: Running, Role: LEADER
I20260805 23:15:00.437305 16431 consensus_queue.cc:237] T 00000000000000000000000000000000 P 984d87a7c9e6457288a6bac46d18f595 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "984d87a7c9e6457288a6bac46d18f595" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 35131 } }
I20260805 23:15:00.437834 16421 sys_catalog.cc:564] T 00000000000000000000000000000000 P 984d87a7c9e6457288a6bac46d18f595 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:15:00.439406 16432 sys_catalog.cc:455] T 00000000000000000000000000000000 P 984d87a7c9e6457288a6bac46d18f595 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "984d87a7c9e6457288a6bac46d18f595" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "984d87a7c9e6457288a6bac46d18f595" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 35131 } } }
I20260805 23:15:00.439356 16433 sys_catalog.cc:455] T 00000000000000000000000000000000 P 984d87a7c9e6457288a6bac46d18f595 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 984d87a7c9e6457288a6bac46d18f595. Latest consensus state: current_term: 1 leader_uuid: "984d87a7c9e6457288a6bac46d18f595" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "984d87a7c9e6457288a6bac46d18f595" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 35131 } } }
I20260805 23:15:00.439622 16433 sys_catalog.cc:458] T 00000000000000000000000000000000 P 984d87a7c9e6457288a6bac46d18f595 [sys.catalog]: This master's current role is: LEADER
I20260805 23:15:00.439939 16432 sys_catalog.cc:458] T 00000000000000000000000000000000 P 984d87a7c9e6457288a6bac46d18f595 [sys.catalog]: This master's current role is: LEADER
I20260805 23:15:00.440023 16444 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:15:00.443779 16444 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:15:00.450109 16444 catalog_manager.cc:1347] Generated new cluster ID: 0635cb6b1ac44868b396693815b23d93
I20260805 23:15:00.450203 16444 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:15:00.471400 16444 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:15:00.471782 16444 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:15:00.480818 16444 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 984d87a7c9e6457288a6bac46d18f595: Generated new TSK 0
I20260805 23:15:00.481738 16444 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260805 23:15:00.487901 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 16352
2026-08-05T23:15:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1474 ms)
[----------] 18 tests from AuthTokenIssuingTest (27047 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971406251336-25971-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 23:15:00 dist-test-slave-ss4n krb5kdc[16453](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 23:15:00 dist-test-slave-ss4n krb5kdc[16453](info): set up 2 sockets
Aug 05 23:15:00 dist-test-slave-ss4n krb5kdc[16453](info): commencing operation
krb5kdc: starting...
W20260805 23:15:01.588229 25971 mini_kdc.cc:121] Time spent starting KDC: real 1.080s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971406251336-25971-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971406251336-25971-0/krb5kdc/test-user.keytab.
Aug 05 23:15:01 dist-test-slave-ss4n krb5kdc[16453](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971701, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T23:15:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:15:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.92.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.92.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.92.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab.
Entry for principal kudu/127.25.92.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab.
I20260805 23:15:01.733824 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971406251336-25971-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971406251336-25971-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971406251336-25971-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.254:35765
--webserver_interface=127.25.92.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab
--principal=kudu/127.25.92.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:34241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.92.254:35765,127.25.92.253:46123,127.25.92.252:39001
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:15:01.853389 16467 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260805 23:15:01.853868 16467 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:15:01.853935 16467 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:15:01.854015 16467 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:15:01.858330 16467 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:15:01.858434 16467 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:15:01.858471 16467 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:15:01.858510 16467 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:15:01.858547 16467 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:15:01.864372 16467 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:34241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971406251336-25971-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971406251336-25971-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.92.254:35765,127.25.92.253:46123,127.25.92.252:39001
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab
--principal=kudu/127.25.92.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.254:35765
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.92.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16467
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971406251336-25971-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:15:01.865666 16467 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:15:01.866775 16467 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:15:01.875872 16473 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:01.875952 16475 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:01.875892 16472 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:01.876133 16467 server_base.cc:1034] running on GCE node
I20260805 23:15:01.876888 16467 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:15:01.878139 16467 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:15:01.879362 16467 hybrid_clock.cc:648] HybridClock initialized: now 1785971701879314 us; error 45 us; skew 500 ppm
Aug 05 23:15:01 dist-test-slave-ss4n krb5kdc[16453](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971701, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:15:01.883356 16467 init.cc:377] Logged in from keytab as kudu/127.25.92.254@KRBTEST.COM (short username kudu)
I20260805 23:15:01.884464 16467 webserver.cc:460] Webserver started at http://127.25.92.254:44245/ using document root <none> and password file <none>
I20260805 23:15:01.885119 16467 fs_manager.cc:359] Metadata directory not provided
I20260805 23:15:01.885195 16467 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:15:01.885563 16467 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:15:01.888144 16467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971406251336-25971-0/minicluster-data/master-0/data/instance:
uuid: "60c980e4dfdd4342a546d2796bd268c8"
format_stamp: "Formatted at 2026-08-05 23:15:01 on dist-test-slave-ss4n"
server_key: "8dd8d894afdbe5148b23b100b65c52f7"
server_key_iv: "d600256622e3a397019de6520bcfb264"
server_key_version: "encryptionkey@0"
I20260805 23:15:01.888855 16467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971406251336-25971-0/minicluster-data/master-0/wal/instance:
uuid: "60c980e4dfdd4342a546d2796bd268c8"
format_stamp: "Formatted at 2026-08-05 23:15:01 on dist-test-slave-ss4n"
server_key: "8dd8d894afdbe5148b23b100b65c52f7"
server_key_iv: "d600256622e3a397019de6520bcfb264"
server_key_version: "encryptionkey@0"
I20260805 23:15:01.893616 16467 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.002s
I20260805 23:15:01.896333 16482 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:01.897809 16467 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:15:01.897976 16467 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971406251336-25971-0/minicluster-data/master-0/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971406251336-25971-0/minicluster-data/master-0/wal
uuid: "60c980e4dfdd4342a546d2796bd268c8"
format_stamp: "Formatted at 2026-08-05 23:15:01 on dist-test-slave-ss4n"
server_key: "8dd8d894afdbe5148b23b100b65c52f7"
server_key_iv: "d600256622e3a397019de6520bcfb264"
server_key_version: "encryptionkey@0"
I20260805 23:15:01.898109 16467 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971406251336-25971-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971406251336-25971-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971406251336-25971-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:15:01.920354 16467 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:15:01.925783 16467 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:15:01.925995 16467 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:15:01.933669 16467 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.254:35765
I20260805 23:15:01.934018 16535 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.254:35765 every 8 connection(s)
I20260805 23:15:01.934718 16467 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
I20260805 23:15:01.937995 16536 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:15:01.941879 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 16467
I20260805 23:15:01.942003 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971406251336-25971-0/minicluster-data/master-0/wal/instance
I20260805 23:15:01.942324 25971 external_mini_cluster.cc:1428] Setting key a7f2f2be85f1cf3ea1099b2a9c7678dd
I20260805 23:15:01.942667 16536 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.92.254" port: 35765 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.25.92.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.92.253@KRBTEST.COM" created.
Aug 05 23:15:01 dist-test-slave-ss4n krb5kdc[16453](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971701, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.254@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM
I20260805 23:15:01.962708 16539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:01.945669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:35765 (local address 127.0.0.1:52288)
0805 23:15:01.946382 (+ 713us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:01.946434 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:01.947426 (+ 992us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:01.948380 (+ 954us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:01.948385 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:01.948783 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:15:01.949073 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:01.949087 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:01.951196 (+ 2109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:01.951199 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:01.952327 (+ 1128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:01.952340 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:01.953184 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:01.953190 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:01.953380 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:01.954094 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:15:01.954112 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:15:01.956639 (+ 2527us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:15:01.958938 (+ 2299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:01.958952 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:01.958969 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:01.959337 (+ 368us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:01.959767 (+ 430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:01.959773 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:01.959783 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:01.959961 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:01.960776 (+ 815us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:15:01.960787 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:15:01.960996 (+ 209us) client_negotiation.cc:769] Sending connection context
0805 23:15:01.961229 (+ 233us) client_negotiation.cc:240] Negotiation successful
0805 23:15:01.961848 (+ 619us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":516,"mutex_wait_us":193,"thread_start_us":183,"threads_started":1}
I20260805 23:15:01.962706 16540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:01.945669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52288 (local address 127.25.92.254:35765)
0805 23:15:01.946278 (+ 609us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:01.946305 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:01.946579 (+ 274us) server_negotiation.cc:407] Connection header received
0805 23:15:01.947782 (+ 1203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:01.947791 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:01.948124 (+ 333us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:01.948380 (+ 256us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:01.949262 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:01.950992 (+ 1730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:01.952484 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:01.952966 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:01.953187 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:01.956830 (+ 3643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:01.956836 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:01.956848 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:01.956871 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:01.958799 (+ 1928us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:01.959514 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:01.959520 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:01.959521 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:01.959652 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:01.960069 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:01.960071 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:01.960073 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:01.960641 (+ 568us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:01.960756 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:01.961315 (+ 559us) server_negotiation.cc:299] Negotiation successful
0805 23:15:01.961742 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":41,"server-negotiator.queue_time_us":405,"spinlock_wait_cycles":52608,"thread_start_us":182,"threads_started":1}
I20260805 23:15:01.965903 16536 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.92.253" port: 46123 } has no permanent_uuid. Determining permanent_uuid...
I20260805 23:15:01.966866 16539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:01.966539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.253:46123 (local address 127.0.0.1:36748)
0805 23:15:01.966694 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:01.966788 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.253:46123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260805 23:15:01.967105 16485 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.25.92.253:46123: connect: Connection refused (error 111)
I20260805 23:15:01.968243 16539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:01.967960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.253:46123 (local address 127.0.0.1:36762)
0805 23:15:01.968121 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:01.968177 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.253:46123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260805 23:15:01.968436 16536 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.92.253:46123: Network error: Client connection negotiation failed: client connection to 127.25.92.253:46123: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.92.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.253.keytab.
Entry for principal kudu/127.25.92.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.253.keytab.
I20260805 23:15:01.978092 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971406251336-25971-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971406251336-25971-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971406251336-25971-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971406251336-25971-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.253:46123
--webserver_interface=127.25.92.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.253.keytab
--principal=kudu/127.25.92.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:34241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.92.254:35765,127.25.92.253:46123,127.25.92.252:39001
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260805 23:15:02.017771 16536 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.92.253" port: 46123 } attempt: 1
I20260805 23:15:02.018883 16539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:02.018471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.253:46123 (local address 127.0.0.1:36768)
0805 23:15:02.018698 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:02.018782 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.253:46123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
I20260805 23:15:02.019816 16539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:02.019557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.253:46123 (local address 127.0.0.1:36778)
0805 23:15:02.019685 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:02.019750 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.253:46123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260805 23:15:02.020053 16536 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.92.253:46123: Network error: Client connection negotiation failed: client connection to 127.25.92.253:46123: connect: Connection refused (error 111)
I20260805 23:15:02.088260 16536 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.92.253" port: 46123 } attempt: 2
I20260805 23:15:02.089318 16539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:02.088941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.253:46123 (local address 127.0.0.1:36794)
0805 23:15:02.089134 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:02.089231 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.253:46123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260805 23:15:02.090143 16539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:02.089881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.253:46123 (local address 127.0.0.1:36802)
0805 23:15:02.090010 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:02.090070 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.253:46123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260805 23:15:02.090305 16536 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.92.253:46123: Network error: Client connection negotiation failed: client connection to 127.25.92.253:46123: connect: Connection refused (error 111)
W20260805 23:15:02.112748 16543 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260805 23:15:02.113145 16543 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:15:02.113224 16543 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:15:02.113310 16543 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:15:02.117647 16543 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:15:02.117744 16543 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:15:02.117779 16543 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:15:02.117820 16543 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:15:02.117856 16543 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:15:02.123824 16543 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:34241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971406251336-25971-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971406251336-25971-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.25.92.254:35765,127.25.92.253:46123,127.25.92.252:39001
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.253.keytab
--principal=kudu/127.25.92.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.253:46123
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971406251336-25971-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.25.92.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16543
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971406251336-25971-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:15:02.125273 16543 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:15:02.126431 16543 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:15:02.135613 16551 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:02.135635 16549 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:02.135699 16548 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:02.135938 16543 server_base.cc:1034] running on GCE node
I20260805 23:15:02.136662 16543 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:15:02.137961 16543 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:15:02.139230 16543 hybrid_clock.cc:648] HybridClock initialized: now 1785971702139223 us; error 36 us; skew 500 ppm
Aug 05 23:15:02 dist-test-slave-ss4n krb5kdc[16453](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971702, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:15:02.142818 16543 init.cc:377] Logged in from keytab as kudu/127.25.92.253@KRBTEST.COM (short username kudu)
I20260805 23:15:02.143971 16543 webserver.cc:460] Webserver started at http://127.25.92.253:45513/ using document root <none> and password file <none>
I20260805 23:15:02.144657 16543 fs_manager.cc:359] Metadata directory not provided
I20260805 23:15:02.144762 16543 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:15:02.145020 16543 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:15:02.147424 16543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971406251336-25971-0/minicluster-data/master-1/data/instance:
uuid: "b654b353a3e94ddfa4f019b14ef9fcef"
format_stamp: "Formatted at 2026-08-05 23:15:02 on dist-test-slave-ss4n"
server_key: "2ea9bb2d45f78039dc42797414efd1a5"
server_key_iv: "c4ba2d27d93ad7fe5df15f00b6a52151"
server_key_version: "encryptionkey@0"
I20260805 23:15:02.148080 16543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971406251336-25971-0/minicluster-data/master-1/wal/instance:
uuid: "b654b353a3e94ddfa4f019b14ef9fcef"
format_stamp: "Formatted at 2026-08-05 23:15:02 on dist-test-slave-ss4n"
server_key: "2ea9bb2d45f78039dc42797414efd1a5"
server_key_iv: "c4ba2d27d93ad7fe5df15f00b6a52151"
server_key_version: "encryptionkey@0"
I20260805 23:15:02.152424 16543 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260805 23:15:02.155526 16558 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:02.156838 16543 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260805 23:15:02.157002 16543 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971406251336-25971-0/minicluster-data/master-1/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971406251336-25971-0/minicluster-data/master-1/wal
uuid: "b654b353a3e94ddfa4f019b14ef9fcef"
format_stamp: "Formatted at 2026-08-05 23:15:02 on dist-test-slave-ss4n"
server_key: "2ea9bb2d45f78039dc42797414efd1a5"
server_key_iv: "c4ba2d27d93ad7fe5df15f00b6a52151"
server_key_version: "encryptionkey@0"
I20260805 23:15:02.157138 16543 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971406251336-25971-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971406251336-25971-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971406251336-25971-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:15:02.180451 16543 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:15:02.181558 16536 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.92.253" port: 46123 } attempt: 3
I20260805 23:15:02.182411 16539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:02.182097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.253:46123 (local address 127.0.0.1:36812)
0805 23:15:02.182261 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:02.182334 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.253:46123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260805 23:15:02.183228 16539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:02.182984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.253:46123 (local address 127.0.0.1:36818)
0805 23:15:02.183101 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:02.183156 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.253:46123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260805 23:15:02.183393 16536 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.92.253:46123: Network error: Client connection negotiation failed: client connection to 127.25.92.253:46123: connect: Connection refused (error 111)
I20260805 23:15:02.184263 16543 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:15:02.184466 16543 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:15:02.192188 16543 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.253:46123
I20260805 23:15:02.192510 16611 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.253:46123 every 8 connection(s)
I20260805 23:15:02.193255 16543 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971406251336-25971-0/minicluster-data/master-1/data/info.pb
I20260805 23:15:02.196357 16612 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:15:02.197813 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 16543
I20260805 23:15:02.197988 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971406251336-25971-0/minicluster-data/master-1/wal/instance
I20260805 23:15:02.198279 25971 external_mini_cluster.cc:1428] Setting key 048391076fddaa13f668535e3ec5fb8f
I20260805 23:15:02.200850 16612 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.92.254" port: 35765 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.25.92.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.92.252@KRBTEST.COM" created.
Aug 05 23:15:02 dist-test-slave-ss4n krb5kdc[16453](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971702, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.253@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM
I20260805 23:15:02.219726 16540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:02.203356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52298 (local address 127.25.92.254:35765)
0805 23:15:02.203563 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:02.203570 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:02.204522 (+ 952us) server_negotiation.cc:407] Connection header received
0805 23:15:02.205689 (+ 1167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:02.205695 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:02.205772 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:02.205895 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:02.207420 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:02.208023 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:02.209803 (+ 1780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:02.210095 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:02.210245 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:02.213634 (+ 3389us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:02.213638 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:02.213640 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:02.213673 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:02.216254 (+ 2581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:02.217077 (+ 823us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:02.217082 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:02.217084 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:02.217213 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:02.217841 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:02.217846 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:02.217848 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:02.218191 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:02.218320 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:02.219262 (+ 942us) server_negotiation.cc:299] Negotiation successful
0805 23:15:02.219497 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260805 23:15:02.221215 16615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:02.203758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:35765 (local address 127.0.0.1:52298)
0805 23:15:02.204316 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:02.204378 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:02.205379 (+ 1001us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:02.206082 (+ 703us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:02.206091 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:02.206539 (+ 448us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:15:02.207194 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:02.207219 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:02.208211 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:02.208214 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:02.209630 (+ 1416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:02.209662 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:02.210299 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:02.210304 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:02.210517 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:02.211300 (+ 783us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:15:02.211332 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:15:02.213299 (+ 1967us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:15:02.216540 (+ 3241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:02.216549 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:02.216580 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:02.216935 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:02.217353 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:02.217357 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:02.217359 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:02.217678 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:02.218399 (+ 721us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:15:02.218408 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:15:02.218872 (+ 464us) client_negotiation.cc:769] Sending connection context
0805 23:15:02.219151 (+ 279us) client_negotiation.cc:240] Negotiation successful
0805 23:15:02.219729 (+ 578us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":169,"threads_started":1}
I20260805 23:15:02.223245 16612 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.92.253" port: 46123 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.92.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.252.keytab.
Entry for principal kudu/127.25.92.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.252.keytab.
Aug 05 23:15:02 dist-test-slave-ss4n krb5kdc[16453](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971702, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.253@KRBTEST.COM for kudu/127.25.92.253@KRBTEST.COM
I20260805 23:15:02.230118 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971406251336-25971-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971406251336-25971-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971406251336-25971-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971406251336-25971-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.252:39001
--webserver_interface=127.25.92.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.252.keytab
--principal=kudu/127.25.92.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:34241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.92.254:35765,127.25.92.253:46123,127.25.92.252:39001
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260805 23:15:02.234586 16615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:02.224007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.253:46123 (local address 127.0.0.1:36828)
0805 23:15:02.224242 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:02.224257 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:02.224346 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:02.224873 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:02.224875 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:02.225053 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:15:02.225153 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:02.225158 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:02.226365 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:02.226367 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:02.226955 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:02.226964 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:02.227806 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:02.227809 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:02.227885 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:02.228430 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:15:02.228442 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:15:02.229949 (+ 1507us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:15:02.232755 (+ 2806us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:02.232761 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:02.232766 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:02.233061 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:02.233388 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:02.233390 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:02.233392 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:02.233473 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:02.234040 (+ 567us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:15:02.234043 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:15:02.234131 (+ 88us) client_negotiation.cc:769] Sending connection context
0805 23:15:02.234244 (+ 113us) client_negotiation.cc:240] Negotiation successful
0805 23:15:02.234388 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147}
I20260805 23:15:02.234599 16617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:02.224097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36828 (local address 127.25.92.253:46123)
0805 23:15:02.224414 (+ 317us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:02.224424 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:02.224446 (+ 22us) server_negotiation.cc:407] Connection header received
0805 23:15:02.224488 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:02.224492 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:02.224718 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:02.224849 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:02.225267 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:02.226246 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:02.227105 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:02.227694 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:02.227791 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:02.230136 (+ 2345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:02.230148 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:02.230163 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:02.230188 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:02.232592 (+ 2404us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:02.233178 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:02.233188 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:02.233193 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:02.233293 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:02.233624 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:02.233627 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:02.233629 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:02.233927 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:02.234084 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:02.234331 (+ 247us) server_negotiation.cc:299] Negotiation successful
0805 23:15:02.234461 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":222,"thread_start_us":113,"threads_started":1}
I20260805 23:15:02.236512 16612 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.92.252" port: 39001 } has no permanent_uuid. Determining permanent_uuid...
I20260805 23:15:02.237349 16615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:02.237065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.252:39001 (local address 127.0.0.1:38994)
0805 23:15:02.237210 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:02.237286 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.252:39001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260805 23:15:02.237671 16559 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.25.92.252:39001: connect: Connection refused (error 111)
I20260805 23:15:02.238799 16615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:02.238521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.252:39001 (local address 127.0.0.1:38996)
0805 23:15:02.238683 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:02.238737 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.252:39001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
W20260805 23:15:02.239116 16612 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.92.252:39001: Network error: Client connection negotiation failed: client connection to 127.25.92.252:39001: connect: Connection refused (error 111)
I20260805 23:15:02.288430 16612 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.92.252" port: 39001 } attempt: 1
I20260805 23:15:02.289450 16615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:02.289070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.252:39001 (local address 127.0.0.1:39008)
0805 23:15:02.289258 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:02.289340 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.252:39001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260805 23:15:02.290334 16615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:02.290077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.252:39001 (local address 127.0.0.1:39022)
0805 23:15:02.290208 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:02.290267 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.252:39001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260805 23:15:02.290526 16612 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.92.252:39001: Network error: Client connection negotiation failed: client connection to 127.25.92.252:39001: connect: Connection refused (error 111)
I20260805 23:15:02.326557 16536 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.92.253" port: 46123 } attempt: 4
Aug 05 23:15:02 dist-test-slave-ss4n krb5kdc[16453](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971701, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.254@KRBTEST.COM for kudu/127.25.92.253@KRBTEST.COM
I20260805 23:15:02.337612 16539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:02.327250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.253:46123 (local address 127.0.0.1:36836)
0805 23:15:02.327506 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:02.327522 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:02.327675 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:02.327988 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:02.327990 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:02.328172 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:15:02.328274 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:02.328280 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:02.329348 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:02.329351 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:02.330079 (+ 728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:02.330090 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:02.330731 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:02.330734 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:02.330830 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:02.331446 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:15:02.331466 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:15:02.332802 (+ 1336us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:15:02.335787 (+ 2985us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:02.335791 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:02.335793 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:02.336034 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:02.336367 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:02.336370 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:02.336371 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:02.336502 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:02.337009 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:15:02.337012 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:15:02.337131 (+ 119us) client_negotiation.cc:769] Sending connection context
0805 23:15:02.337230 (+ 99us) client_negotiation.cc:240] Negotiation successful
0805 23:15:02.337370 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137}
I20260805 23:15:02.337644 16617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:02.327384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36836 (local address 127.25.92.253:46123)
0805 23:15:02.327573 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:02.327578 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:02.327599 (+ 21us) server_negotiation.cc:407] Connection header received
0805 23:15:02.327795 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:02.327799 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:02.327856 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:02.327947 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:02.328401 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:02.329217 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:02.330216 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:02.330576 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:02.330698 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:02.332961 (+ 2263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:02.332965 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:02.332967 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:02.332997 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:02.335622 (+ 2625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:02.336153 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:02.336157 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:02.336159 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:02.336256 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:02.336619 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:02.336622 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:02.336625 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:02.336872 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:02.336983 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:02.337311 (+ 328us) server_negotiation.cc:299] Negotiation successful
0805 23:15:02.337450 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260805 23:15:02.338735 16536 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.92.252" port: 39001 } has no permanent_uuid. Determining permanent_uuid...
I20260805 23:15:02.339522 16539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:02.339264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.252:39001 (local address 127.0.0.1:39036)
0805 23:15:02.339397 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:02.339459 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.252:39001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260805 23:15:02.340235 16539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:02.340012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.252:39001 (local address 127.0.0.1:39038)
0805 23:15:02.340121 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:02.340177 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.252:39001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260805 23:15:02.340371 16536 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.92.252:39001: Network error: Client connection negotiation failed: client connection to 127.25.92.252:39001: connect: Connection refused (error 111)
I20260805 23:15:02.358708 16612 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.92.252" port: 39001 } attempt: 2
I20260805 23:15:02.359684 16615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:02.359273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.252:39001 (local address 127.0.0.1:39054)
0805 23:15:02.359501 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:02.359595 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.252:39001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
W20260805 23:15:02.359674 16618 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260805 23:15:02.360126 16618 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:15:02.360184 16618 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:15:02.360255 16618 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260805 23:15:02.360960 16615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:02.360556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.252:39001 (local address 127.0.0.1:39070)
0805 23:15:02.360744 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:02.360816 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.252:39001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260805 23:15:02.361230 16612 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.92.252:39001: Network error: Client connection negotiation failed: client connection to 127.25.92.252:39001: connect: Connection refused (error 111)
W20260805 23:15:02.365666 16618 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:15:02.365749 16618 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:15:02.365773 16618 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:15:02.365792 16618 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:15:02.365834 16618 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:15:02.372470 16618 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:34241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971406251336-25971-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971406251336-25971-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.25.92.254:35765,127.25.92.253:46123,127.25.92.252:39001
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.252.keytab
--principal=kudu/127.25.92.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.252:39001
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971406251336-25971-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.25.92.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16618
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971406251336-25971-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:15:02.373876 16618 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:15:02.375047 16618 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:15:02.385072 16625 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:02.385206 16628 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:02.385216 16626 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:02.385360 16618 server_base.cc:1034] running on GCE node
I20260805 23:15:02.386070 16618 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:15:02.387411 16618 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:15:02.388686 16618 hybrid_clock.cc:648] HybridClock initialized: now 1785971702388668 us; error 30 us; skew 500 ppm
Aug 05 23:15:02 dist-test-slave-ss4n krb5kdc[16453](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971702, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:15:02.392014 16618 init.cc:377] Logged in from keytab as kudu/127.25.92.252@KRBTEST.COM (short username kudu)
I20260805 23:15:02.393085 16618 webserver.cc:460] Webserver started at http://127.25.92.252:40569/ using document root <none> and password file <none>
I20260805 23:15:02.393785 16618 fs_manager.cc:359] Metadata directory not provided
I20260805 23:15:02.393888 16618 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:15:02.394145 16618 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:15:02.396486 16618 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971406251336-25971-0/minicluster-data/master-2/data/instance:
uuid: "791b9cac4c3445f6afac29049b1ec139"
format_stamp: "Formatted at 2026-08-05 23:15:02 on dist-test-slave-ss4n"
server_key: "d128bb16d3b8a13dbb273c329eabdf6a"
server_key_iv: "8dcbfd00b83dce8423539b8eb533a45e"
server_key_version: "encryptionkey@0"
I20260805 23:15:02.397176 16618 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971406251336-25971-0/minicluster-data/master-2/wal/instance:
uuid: "791b9cac4c3445f6afac29049b1ec139"
format_stamp: "Formatted at 2026-08-05 23:15:02 on dist-test-slave-ss4n"
server_key: "d128bb16d3b8a13dbb273c329eabdf6a"
server_key_iv: "8dcbfd00b83dce8423539b8eb533a45e"
server_key_version: "encryptionkey@0"
I20260805 23:15:02.399551 16536 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.92.252" port: 39001 } attempt: 1
I20260805 23:15:02.400600 16539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:02.400190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.252:39001 (local address 127.0.0.1:39074)
0805 23:15:02.400343 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:02.400418 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.252:39001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260805 23:15:02.401631 16539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:02.401329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.252:39001 (local address 127.0.0.1:39078)
0805 23:15:02.401467 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:02.401568 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.252:39001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260805 23:15:02.401796 16536 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.92.252:39001: Network error: Client connection negotiation failed: client connection to 127.25.92.252:39001: connect: Connection refused (error 111)
I20260805 23:15:02.402029 16618 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260805 23:15:02.404982 16635 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:02.406975 16618 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260805 23:15:02.407146 16618 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971406251336-25971-0/minicluster-data/master-2/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971406251336-25971-0/minicluster-data/master-2/wal
uuid: "791b9cac4c3445f6afac29049b1ec139"
format_stamp: "Formatted at 2026-08-05 23:15:02 on dist-test-slave-ss4n"
server_key: "d128bb16d3b8a13dbb273c329eabdf6a"
server_key_iv: "8dcbfd00b83dce8423539b8eb533a45e"
server_key_version: "encryptionkey@0"
I20260805 23:15:02.407281 16618 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971406251336-25971-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971406251336-25971-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971406251336-25971-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:15:02.433179 16618 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:15:02.436874 16618 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:15:02.437107 16618 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:15:02.444800 16618 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.252:39001
I20260805 23:15:02.445133 16687 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.252:39001 every 8 connection(s)
I20260805 23:15:02.445938 16618 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971406251336-25971-0/minicluster-data/master-2/data/info.pb
I20260805 23:15:02.448899 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 16618
I20260805 23:15:02.448930 16688 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:15:02.449059 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971406251336-25971-0/minicluster-data/master-2/wal/instance
I20260805 23:15:02.449371 25971 external_mini_cluster.cc:1428] Setting key fb02913cf9928b17910d1618b481f540
I20260805 23:15:02.452453 16612 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.92.252" port: 39001 } attempt: 3
I20260805 23:15:02.454206 16688 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.92.254" port: 35765 } has no permanent_uuid. Determining permanent_uuid...
Aug 05 23:15:02 dist-test-slave-ss4n krb5kdc[16453](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971702, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.253@KRBTEST.COM for kudu/127.25.92.252@KRBTEST.COM
Aug 05 23:15:02 dist-test-slave-ss4n krb5kdc[16453](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971701, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.92.252@KRBTEST.COM
Aug 05 23:15:02 dist-test-slave-ss4n krb5kdc[16453](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971702, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.252@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM
I20260805 23:15:02.464697 16615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:02.452973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.252:39001 (local address 127.0.0.1:39106)
0805 23:15:02.453086 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:02.453103 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:02.453182 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:02.453646 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:02.453649 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:02.453841 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:15:02.453953 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:02.453960 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:02.455639 (+ 1679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:02.455643 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:02.456481 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:02.456494 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:02.457657 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:02.457660 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:02.457761 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:02.458267 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:15:02.458283 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:15:02.459647 (+ 1364us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:15:02.462395 (+ 2748us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:02.462400 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:02.462404 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:02.462701 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:02.463228 (+ 527us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:02.463233 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:02.463237 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:02.463361 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:02.464095 (+ 734us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:15:02.464099 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:15:02.464209 (+ 110us) client_negotiation.cc:769] Sending connection context
0805 23:15:02.464324 (+ 115us) client_negotiation.cc:240] Negotiation successful
0805 23:15:02.464483 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260805 23:15:02.466401 16691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:02.450961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39094 (local address 127.25.92.252:39001)
0805 23:15:02.451430 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:02.451445 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:02.451484 (+ 39us) server_negotiation.cc:407] Connection header received
0805 23:15:02.452253 (+ 769us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:02.452272 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:02.452699 (+ 427us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:02.453350 (+ 651us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:02.454152 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:02.455415 (+ 1263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:02.456639 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:02.457526 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:02.457795 (+ 269us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:02.461611 (+ 3816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:02.461615 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:02.461618 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:02.461641 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:02.463598 (+ 1957us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:02.464125 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:02.464128 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:02.464130 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:02.464211 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:02.464515 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:02.464518 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:02.464520 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:02.464707 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:02.465004 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:02.465924 (+ 920us) server_negotiation.cc:299] Negotiation successful
0805 23:15:02.466046 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":159,"server-negotiator.queue_time_us":321,"spinlock_wait_cycles":1792,"thread_start_us":136,"threads_started":1}
I20260805 23:15:02.466413 16692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:02.453035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39106 (local address 127.25.92.252:39001)
0805 23:15:02.453352 (+ 317us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:02.453359 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:02.453374 (+ 15us) server_negotiation.cc:407] Connection header received
0805 23:15:02.453410 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:02.453413 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:02.453454 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:02.454083 (+ 629us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:02.454189 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:02.455462 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:02.456639 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:02.457526 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:02.457775 (+ 249us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:02.459958 (+ 2183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:02.459967 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:02.459982 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:02.460042 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:02.462264 (+ 2222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:02.462819 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:02.462829 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:02.462854 (+ 25us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:02.463085 (+ 231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:02.463481 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:02.463484 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:02.463488 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:02.463974 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:02.464123 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:02.465028 (+ 905us) server_negotiation.cc:299] Negotiation successful
0805 23:15:02.465319 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":382,"server-negotiator.queue_time_us":246,"spinlock_wait_cycles":5376,"thread_start_us":137,"threads_started":1}
I20260805 23:15:02.468623 16540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:02.456278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52304 (local address 127.25.92.254:35765)
0805 23:15:02.456412 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:02.456417 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:02.456794 (+ 377us) server_negotiation.cc:407] Connection header received
0805 23:15:02.457012 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:02.457021 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:02.457074 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:02.457168 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:02.458548 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:02.459504 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:02.461004 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:02.461384 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:02.461569 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:02.464816 (+ 3247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:02.464820 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:02.464822 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:02.464853 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:02.466670 (+ 1817us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:02.467222 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:02.467226 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:02.467228 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:02.467325 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:02.467624 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:02.467628 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:02.467630 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:02.467829 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:02.467944 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:02.468308 (+ 364us) server_negotiation.cc:299] Negotiation successful
0805 23:15:02.468444 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260805 23:15:02.469237 16693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:02.456205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:35765 (local address 127.0.0.1:52304)
0805 23:15:02.456661 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:02.456714 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:02.456888 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:02.457219 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:02.457224 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:02.457731 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:15:02.458345 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:02.458354 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:02.460182 (+ 1828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:02.460185 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:02.460868 (+ 683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:02.460878 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:02.461552 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:02.461556 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:02.461675 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:02.462272 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:15:02.462287 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:15:02.464667 (+ 2380us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:15:02.466766 (+ 2099us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:02.466776 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:02.466784 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:02.467091 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:02.467441 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:02.467444 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:02.467446 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:02.467528 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:02.467942 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:15:02.467949 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:15:02.468061 (+ 112us) client_negotiation.cc:769] Sending connection context
0805 23:15:02.468841 (+ 780us) client_negotiation.cc:240] Negotiation successful
0805 23:15:02.469034 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"mutex_wait_us":418,"thread_start_us":114,"threads_started":1}
I20260805 23:15:02.469559 16536 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.92.252" port: 39001 } attempt: 2
I20260805 23:15:02.471045 16688 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.92.253" port: 46123 } has no permanent_uuid. Determining permanent_uuid...
I20260805 23:15:02.471454 16612 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b654b353a3e94ddfa4f019b14ef9fcef: Bootstrap starting.
Aug 05 23:15:02 dist-test-slave-ss4n krb5kdc[16453](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971701, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM
I20260805 23:15:02.475092 16612 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b654b353a3e94ddfa4f019b14ef9fcef: Neither blocks nor log segments found. Creating new log.
I20260805 23:15:02.476127 16612 log.cc:824] T 00000000000000000000000000000000 P b654b353a3e94ddfa4f019b14ef9fcef: Log is configured to *not* fsync() on all Append() calls
Aug 05 23:15:02 dist-test-slave-ss4n krb5kdc[16453](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971702, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.252@KRBTEST.COM for kudu/127.25.92.253@KRBTEST.COM
Aug 05 23:15:02 dist-test-slave-ss4n krb5kdc[16453](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971701, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.254@KRBTEST.COM for kudu/127.25.92.252@KRBTEST.COM
I20260805 23:15:02.478686 16612 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b654b353a3e94ddfa4f019b14ef9fcef: No bootstrap required, opened a new log
I20260805 23:15:02.479838 16540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:02.469032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52316 (local address 127.25.92.254:35765)
0805 23:15:02.469154 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:02.469163 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:02.469173 (+ 10us) server_negotiation.cc:407] Connection header received
0805 23:15:02.469201 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:02.469203 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:02.469235 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:02.469320 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:02.470361 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:02.470911 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:02.471966 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:02.472198 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:02.472309 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:02.475255 (+ 2946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:02.475259 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:02.475262 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:02.475292 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:02.477041 (+ 1749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:02.477691 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:02.477695 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:02.477697 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:02.477773 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:02.478108 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:02.478112 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:02.478114 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:02.478387 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:02.478620 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:02.479496 (+ 876us) server_negotiation.cc:299] Negotiation successful
0805 23:15:02.479621 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260805 23:15:02.482223 16617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:02.471750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36838 (local address 127.25.92.253:46123)
0805 23:15:02.471868 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:02.471871 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:02.471882 (+ 11us) server_negotiation.cc:407] Connection header received
0805 23:15:02.471912 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:02.471914 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:02.471947 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:02.472036 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:02.472439 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:02.473138 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:02.474249 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:02.474546 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:02.474633 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:02.477933 (+ 3300us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:02.477937 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:02.477939 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:02.477966 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:02.480178 (+ 2212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:02.480743 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:02.480747 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:02.480749 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:02.480834 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:02.481161 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:02.481164 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:02.481166 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:02.481409 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:02.481521 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:02.481896 (+ 375us) server_negotiation.cc:299] Negotiation successful
0805 23:15:02.482035 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260805 23:15:02.482733 16693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:02.471541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.253:46123 (local address 127.0.0.1:36838)
0805 23:15:02.471664 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:02.471678 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:02.471751 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:02.472052 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:02.472055 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:02.472234 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:15:02.472337 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:02.472347 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:02.473260 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:02.473264 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:02.474111 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:02.474122 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:02.474783 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:02.474787 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:02.474888 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:02.475424 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:15:02.475442 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:15:02.477772 (+ 2330us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:15:02.480318 (+ 2546us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:02.480321 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:02.480323 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:02.480625 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:02.480961 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:02.480965 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:02.480967 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:02.481056 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:02.481606 (+ 550us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:15:02.481609 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:15:02.481706 (+ 97us) client_negotiation.cc:769] Sending connection context
0805 23:15:02.482399 (+ 693us) client_negotiation.cc:240] Negotiation successful
0805 23:15:02.482552 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260805 23:15:02.482826 16539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:02.469965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.252:39001 (local address 127.0.0.1:39114)
0805 23:15:02.472206 (+ 2241us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:02.472223 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:02.472299 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:02.473807 (+ 1508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:02.473813 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:02.474001 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:15:02.474122 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:02.474129 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:02.475382 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:02.475385 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:02.476284 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:02.476299 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:02.476971 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:02.476976 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:02.477068 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:02.477522 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:15:02.477538 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:15:02.478950 (+ 1412us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:15:02.480990 (+ 2040us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:02.480994 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:02.480996 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:02.481248 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:02.481662 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:02.481665 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:02.481667 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:02.481755 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:02.482256 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:15:02.482259 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:15:02.482371 (+ 112us) client_negotiation.cc:769] Sending connection context
0805 23:15:02.482464 (+ 93us) client_negotiation.cc:240] Negotiation successful
0805 23:15:02.482607 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2163}
I20260805 23:15:02.483457 16688 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.92.252" port: 39001 } has no permanent_uuid. Determining permanent_uuid...
I20260805 23:15:02.483696 16612 raft_consensus.cc:348] T 00000000000000000000000000000000 P b654b353a3e94ddfa4f019b14ef9fcef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60c980e4dfdd4342a546d2796bd268c8" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 35765 } } peers { permanent_uuid: "b654b353a3e94ddfa4f019b14ef9fcef" member_type: VOTER last_known_addr { host: "127.25.92.253" port: 46123 } } peers { permanent_uuid: "791b9cac4c3445f6afac29049b1ec139" member_type: VOTER last_known_addr { host: "127.25.92.252" port: 39001 } }
I20260805 23:15:02.483973 16612 raft_consensus.cc:729] T 00000000000000000000000000000000 P b654b353a3e94ddfa4f019b14ef9fcef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b654b353a3e94ddfa4f019b14ef9fcef, State: Initialized, Role: FOLLOWER
I20260805 23:15:02.484357 16612 consensus_queue.cc:260] T 00000000000000000000000000000000 P b654b353a3e94ddfa4f019b14ef9fcef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60c980e4dfdd4342a546d2796bd268c8" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 35765 } } peers { permanent_uuid: "b654b353a3e94ddfa4f019b14ef9fcef" member_type: VOTER last_known_addr { host: "127.25.92.253" port: 46123 } } peers { permanent_uuid: "791b9cac4c3445f6afac29049b1ec139" member_type: VOTER last_known_addr { host: "127.25.92.252" port: 39001 } }
I20260805 23:15:02.484534 16691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:02.470188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39114 (local address 127.25.92.252:39001)
0805 23:15:02.470442 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:02.470446 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:02.473569 (+ 3123us) server_negotiation.cc:407] Connection header received
0805 23:15:02.473618 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:02.473622 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:02.473673 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:02.474231 (+ 558us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:02.474337 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:02.475266 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:02.476444 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:02.476826 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:02.477282 (+ 456us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:02.479095 (+ 1813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:02.479099 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:02.479101 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:02.479125 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:02.480844 (+ 1719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:02.481378 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:02.481383 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:02.481387 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:02.481546 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:02.481885 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:02.481889 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:02.481894 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:02.482108 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:02.484204 (+ 2096us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:02.484279 (+ 75us) server_negotiation.cc:299] Negotiation successful
0805 23:15:02.484385 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184}
I20260805 23:15:02.485893 16612 sys_catalog.cc:564] T 00000000000000000000000000000000 P b654b353a3e94ddfa4f019b14ef9fcef [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:15:02.487787 16536 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60c980e4dfdd4342a546d2796bd268c8: Bootstrap starting.
Aug 05 23:15:02 dist-test-slave-ss4n krb5kdc[16453](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971701, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.92.253@KRBTEST.COM
Aug 05 23:15:02 dist-test-slave-ss4n krb5kdc[16453](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971702, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.252@KRBTEST.COM for kudu/127.25.92.252@KRBTEST.COM
I20260805 23:15:02.490969 16536 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 60c980e4dfdd4342a546d2796bd268c8: Neither blocks nor log segments found. Creating new log.
I20260805 23:15:02.491937 16536 log.cc:824] T 00000000000000000000000000000000 P 60c980e4dfdd4342a546d2796bd268c8: Log is configured to *not* fsync() on all Append() calls
I20260805 23:15:02.493228 16693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:02.483939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.252:39001 (local address 127.0.0.1:39118)
0805 23:15:02.484711 (+ 772us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:02.484725 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:02.484824 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:02.485122 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:02.485124 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:02.485329 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:15:02.485415 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:02.485421 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:02.486365 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:02.486368 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:02.487078 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:02.487088 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:02.487732 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:02.487735 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:02.487814 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:02.488306 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:15:02.488320 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:15:02.489800 (+ 1480us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:15:02.491668 (+ 1868us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:02.491671 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:02.491673 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:02.491911 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:02.492212 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:02.492215 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:02.492216 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:02.492296 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:02.492736 (+ 440us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:15:02.492738 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:15:02.492794 (+ 56us) client_negotiation.cc:769] Sending connection context
0805 23:15:02.492906 (+ 112us) client_negotiation.cc:240] Negotiation successful
0805 23:15:02.493054 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":696}
I20260805 23:15:02.493247 16692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:02.484023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39118 (local address 127.25.92.252:39001)
0805 23:15:02.484128 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:02.484132 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:02.484771 (+ 639us) server_negotiation.cc:407] Connection header received
0805 23:15:02.484916 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:02.484919 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:02.484954 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:02.485001 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:02.485579 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:02.486248 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:02.487197 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:02.487525 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:02.487602 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:02.489942 (+ 2340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:02.489945 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:02.489948 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:02.489970 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:02.491553 (+ 1583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:02.492028 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:02.492031 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:02.492033 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:02.492112 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:02.492387 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:02.492390 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:02.492392 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:02.492614 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:02.492692 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:02.492949 (+ 257us) server_negotiation.cc:299] Negotiation successful
0805 23:15:02.493115 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":58,"server-negotiator.queue_time_us":42}
I20260805 23:15:02.490147 16543 master_runner.cc:186] Error getting master registration for 127.25.92.254:35765: OK, instance_id { permanent_uuid: "60c980e4dfdd4342a546d2796bd268c8" instance_seqno: 1785971701931487 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260805 23:15:02.494721 16543 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260805 23:15:02.496615 16688 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 791b9cac4c3445f6afac29049b1ec139: Bootstrap starting.
I20260805 23:15:02.498036 16695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:02.481298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36842 (local address 127.25.92.253:46123)
0805 23:15:02.482110 (+ 812us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:02.482120 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:02.482135 (+ 15us) server_negotiation.cc:407] Connection header received
0805 23:15:02.482174 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:02.482206 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:02.482258 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:02.483155 (+ 897us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:02.483310 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:02.484085 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:02.485358 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:02.485736 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:02.489455 (+ 3719us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:02.489612 (+ 157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:02.489615 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:02.489618 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:02.489634 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:02.491956 (+ 2322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:02.492481 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:02.492484 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:02.492486 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:02.492600 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:02.492905 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:02.492908 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:02.492910 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:02.493108 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:02.494009 (+ 901us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:02.497690 (+ 3681us) server_negotiation.cc:299] Negotiation successful
0805 23:15:02.497850 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":742,"thread_start_us":174,"threads_started":1}
W20260805 23:15:02.498553 16707 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b654b353a3e94ddfa4f019b14ef9fcef: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260805 23:15:02.498629 16707 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260805 23:15:02.498816 16696 sys_catalog.cc:455] T 00000000000000000000000000000000 P b654b353a3e94ddfa4f019b14ef9fcef [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60c980e4dfdd4342a546d2796bd268c8" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 35765 } } peers { permanent_uuid: "b654b353a3e94ddfa4f019b14ef9fcef" member_type: VOTER last_known_addr { host: "127.25.92.253" port: 46123 } } peers { permanent_uuid: "791b9cac4c3445f6afac29049b1ec139" member_type: VOTER last_known_addr { host: "127.25.92.252" port: 39001 } } }
I20260805 23:15:02.498942 16696 sys_catalog.cc:458] T 00000000000000000000000000000000 P b654b353a3e94ddfa4f019b14ef9fcef [sys.catalog]: This master's current role is: FOLLOWER
I20260805 23:15:02.499413 16688 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 791b9cac4c3445f6afac29049b1ec139: Neither blocks nor log segments found. Creating new log.
I20260805 23:15:02.500288 16688 log.cc:824] T 00000000000000000000000000000000 P 791b9cac4c3445f6afac29049b1ec139: Log is configured to *not* fsync() on all Append() calls
I20260805 23:15:02.501418 16536 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60c980e4dfdd4342a546d2796bd268c8: No bootstrap required, opened a new log
I20260805 23:15:02.501770 25971 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260805 23:15:02.502991 16688 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 791b9cac4c3445f6afac29049b1ec139: No bootstrap required, opened a new log
I20260805 23:15:02.503255 16510 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "60c980e4dfdd4342a546d2796bd268c8"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52316
I20260805 23:15:02.506340 16536 raft_consensus.cc:348] T 00000000000000000000000000000000 P 60c980e4dfdd4342a546d2796bd268c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60c980e4dfdd4342a546d2796bd268c8" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 35765 } } peers { permanent_uuid: "b654b353a3e94ddfa4f019b14ef9fcef" member_type: VOTER last_known_addr { host: "127.25.92.253" port: 46123 } } peers { permanent_uuid: "791b9cac4c3445f6afac29049b1ec139" member_type: VOTER last_known_addr { host: "127.25.92.252" port: 39001 } }
I20260805 23:15:02.506587 16536 raft_consensus.cc:729] T 00000000000000000000000000000000 P 60c980e4dfdd4342a546d2796bd268c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60c980e4dfdd4342a546d2796bd268c8, State: Initialized, Role: FOLLOWER
I20260805 23:15:02.506939 16536 consensus_queue.cc:260] T 00000000000000000000000000000000 P 60c980e4dfdd4342a546d2796bd268c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60c980e4dfdd4342a546d2796bd268c8" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 35765 } } peers { permanent_uuid: "b654b353a3e94ddfa4f019b14ef9fcef" member_type: VOTER last_known_addr { host: "127.25.92.253" port: 46123 } } peers { permanent_uuid: "791b9cac4c3445f6afac29049b1ec139" member_type: VOTER last_known_addr { host: "127.25.92.252" port: 39001 } }
I20260805 23:15:02.507012 16688 raft_consensus.cc:348] T 00000000000000000000000000000000 P 791b9cac4c3445f6afac29049b1ec139 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60c980e4dfdd4342a546d2796bd268c8" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 35765 } } peers { permanent_uuid: "b654b353a3e94ddfa4f019b14ef9fcef" member_type: VOTER last_known_addr { host: "127.25.92.253" port: 46123 } } peers { permanent_uuid: "791b9cac4c3445f6afac29049b1ec139" member_type: VOTER last_known_addr { host: "127.25.92.252" port: 39001 } }
I20260805 23:15:02.507256 16688 raft_consensus.cc:729] T 00000000000000000000000000000000 P 791b9cac4c3445f6afac29049b1ec139 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 791b9cac4c3445f6afac29049b1ec139, State: Initialized, Role: FOLLOWER
I20260805 23:15:02.507633 16688 consensus_queue.cc:260] T 00000000000000000000000000000000 P 791b9cac4c3445f6afac29049b1ec139 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60c980e4dfdd4342a546d2796bd268c8" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 35765 } } peers { permanent_uuid: "b654b353a3e94ddfa4f019b14ef9fcef" member_type: VOTER last_known_addr { host: "127.25.92.253" port: 46123 } } peers { permanent_uuid: "791b9cac4c3445f6afac29049b1ec139" member_type: VOTER last_known_addr { host: "127.25.92.252" port: 39001 } }
I20260805 23:15:02.507704 16711 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60c980e4dfdd4342a546d2796bd268c8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60c980e4dfdd4342a546d2796bd268c8" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 35765 } } peers { permanent_uuid: "b654b353a3e94ddfa4f019b14ef9fcef" member_type: VOTER last_known_addr { host: "127.25.92.253" port: 46123 } } peers { permanent_uuid: "791b9cac4c3445f6afac29049b1ec139" member_type: VOTER last_known_addr { host: "127.25.92.252" port: 39001 } } }
I20260805 23:15:02.508088 16711 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60c980e4dfdd4342a546d2796bd268c8 [sys.catalog]: This master's current role is: FOLLOWER
I20260805 23:15:02.508463 16536 sys_catalog.cc:564] T 00000000000000000000000000000000 P 60c980e4dfdd4342a546d2796bd268c8 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:15:02.508523 16510 raft_consensus.cc:482] T 00000000000000000000000000000000 P 60c980e4dfdd4342a546d2796bd268c8 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260805 23:15:02.508658 16510 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 60c980e4dfdd4342a546d2796bd268c8 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:15:02.508874 16688 sys_catalog.cc:564] T 00000000000000000000000000000000 P 791b9cac4c3445f6afac29049b1ec139 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:15:02.510162 16510 raft_consensus.cc:504] T 00000000000000000000000000000000 P 60c980e4dfdd4342a546d2796bd268c8 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60c980e4dfdd4342a546d2796bd268c8" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 35765 } } peers { permanent_uuid: "b654b353a3e94ddfa4f019b14ef9fcef" member_type: VOTER last_known_addr { host: "127.25.92.253" port: 46123 } } peers { permanent_uuid: "791b9cac4c3445f6afac29049b1ec139" member_type: VOTER last_known_addr { host: "127.25.92.252" port: 39001 } }
I20260805 23:15:02.511193 16510 leader_election.cc:288] T 00000000000000000000000000000000 P 60c980e4dfdd4342a546d2796bd268c8 [CANDIDATE]: Term 1 election: Requested vote from peers b654b353a3e94ddfa4f019b14ef9fcef (127.25.92.253:46123), 791b9cac4c3445f6afac29049b1ec139 (127.25.92.252:39001)
I20260805 23:15:02.511938 16662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "60c980e4dfdd4342a546d2796bd268c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "791b9cac4c3445f6afac29049b1ec139"
I20260805 23:15:02.512199 16662 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 791b9cac4c3445f6afac29049b1ec139 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:15:02.512151 16585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "60c980e4dfdd4342a546d2796bd268c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "b654b353a3e94ddfa4f019b14ef9fcef"
I20260805 23:15:02.512393 16585 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b654b353a3e94ddfa4f019b14ef9fcef [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:15:02.513706 16662 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 791b9cac4c3445f6afac29049b1ec139 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60c980e4dfdd4342a546d2796bd268c8 in term 1.
I20260805 23:15:02.513764 16585 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b654b353a3e94ddfa4f019b14ef9fcef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60c980e4dfdd4342a546d2796bd268c8 in term 1.
I20260805 23:15:02.514772 16484 leader_election.cc:302] T 00000000000000000000000000000000 P 60c980e4dfdd4342a546d2796bd268c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60c980e4dfdd4342a546d2796bd268c8, 791b9cac4c3445f6afac29049b1ec139; no voters:
I20260805 23:15:02.515076 16711 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 60c980e4dfdd4342a546d2796bd268c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:15:02.515393 16711 raft_consensus.cc:686] T 00000000000000000000000000000000 P 60c980e4dfdd4342a546d2796bd268c8 [term 1 LEADER]: Becoming Leader. State: Replica: 60c980e4dfdd4342a546d2796bd268c8, State: Running, Role: LEADER
I20260805 23:15:02.516816 16711 consensus_queue.cc:237] T 00000000000000000000000000000000 P 60c980e4dfdd4342a546d2796bd268c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60c980e4dfdd4342a546d2796bd268c8" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 35765 } } peers { permanent_uuid: "b654b353a3e94ddfa4f019b14ef9fcef" member_type: VOTER last_known_addr { host: "127.25.92.253" port: 46123 } } peers { permanent_uuid: "791b9cac4c3445f6afac29049b1ec139" member_type: VOTER last_known_addr { host: "127.25.92.252" port: 39001 } }
I20260805 23:15:02.517704 16712 sys_catalog.cc:455] T 00000000000000000000000000000000 P 791b9cac4c3445f6afac29049b1ec139 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60c980e4dfdd4342a546d2796bd268c8" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 35765 } } peers { permanent_uuid: "b654b353a3e94ddfa4f019b14ef9fcef" member_type: VOTER last_known_addr { host: "127.25.92.253" port: 46123 } } peers { permanent_uuid: "791b9cac4c3445f6afac29049b1ec139" member_type: VOTER last_known_addr { host: "127.25.92.252" port: 39001 } } }
I20260805 23:15:02.517843 16712 sys_catalog.cc:458] T 00000000000000000000000000000000 P 791b9cac4c3445f6afac29049b1ec139 [sys.catalog]: This master's current role is: FOLLOWER
I20260805 23:15:02.518918 16711 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60c980e4dfdd4342a546d2796bd268c8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 60c980e4dfdd4342a546d2796bd268c8. Latest consensus state: current_term: 1 leader_uuid: "60c980e4dfdd4342a546d2796bd268c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60c980e4dfdd4342a546d2796bd268c8" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 35765 } } peers { permanent_uuid: "b654b353a3e94ddfa4f019b14ef9fcef" member_type: VOTER last_known_addr { host: "127.25.92.253" port: 46123 } } peers { permanent_uuid: "791b9cac4c3445f6afac29049b1ec139" member_type: VOTER last_known_addr { host: "127.25.92.252" port: 39001 } } }
I20260805 23:15:02.519037 16711 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60c980e4dfdd4342a546d2796bd268c8 [sys.catalog]: This master's current role is: LEADER
I20260805 23:15:02.520622 16467 master_runner.cc:263] No leader master found from master 60c980e4dfdd4342a546d2796bd268c8
I20260805 23:15:02.520695 16467 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260805 23:15:02.522688 16729 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 60c980e4dfdd4342a546d2796bd268c8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260805 23:15:02.522845 16729 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260805 23:15:02.526639 16739 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 791b9cac4c3445f6afac29049b1ec139: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260805 23:15:02.526739 16739 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260805 23:15:02.534198 16692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:02.522550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39124 (local address 127.25.92.252:39001)
0805 23:15:02.522672 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:02.522676 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:02.523936 (+ 1260us) server_negotiation.cc:407] Connection header received
0805 23:15:02.524150 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:02.524153 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:02.524204 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:02.524257 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:02.525351 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:02.526185 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:02.527547 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:02.527881 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:02.527958 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:02.530056 (+ 2098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:02.530072 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:02.530075 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:02.530106 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:02.531948 (+ 1842us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:02.532695 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:02.532698 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:02.532701 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:02.532791 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:02.533122 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:02.533126 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:02.533128 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:02.533349 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:02.533469 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:02.533878 (+ 409us) server_negotiation.cc:299] Negotiation successful
0805 23:15:02.534005 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260805 23:15:02.541543 16540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:02.523469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52324 (local address 127.25.92.254:35765)
0805 23:15:02.523576 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:02.523580 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:02.527280 (+ 3700us) server_negotiation.cc:407] Connection header received
0805 23:15:02.528748 (+ 1468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:02.528751 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:02.528814 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:02.528873 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:02.529837 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:02.530627 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:02.533882 (+ 3255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:02.534260 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:02.534348 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:02.537624 (+ 3276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:02.537630 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:02.537634 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:02.537675 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:02.539522 (+ 1847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:02.540043 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:02.540047 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:02.540049 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:02.540149 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:02.540450 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:02.540453 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:02.540455 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:02.540689 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:02.540799 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:02.541190 (+ 391us) server_negotiation.cc:299] Negotiation successful
0805 23:15:02.541324 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260805 23:15:02.543883 16695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:02.522473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36854 (local address 127.25.92.253:46123)
0805 23:15:02.522666 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:02.522672 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:02.527046 (+ 4374us) server_negotiation.cc:407] Connection header received
0805 23:15:02.527571 (+ 525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:02.527575 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:02.527631 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:02.527685 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:02.531860 (+ 4175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:02.532679 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:02.535205 (+ 2526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:02.535563 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:02.535645 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:02.539346 (+ 3701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:02.539354 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:02.539359 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:02.539393 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:02.541203 (+ 1810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:02.542375 (+ 1172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:02.542379 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:02.542381 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:02.542489 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:02.542831 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:02.542835 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:02.542837 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:02.543095 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:02.543235 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:02.543576 (+ 341us) server_negotiation.cc:299] Negotiation successful
0805 23:15:02.543719 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118}
I20260805 23:15:02.551561 16742 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:15:02.552553 16742 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:15:02.558730 16662 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 791b9cac4c3445f6afac29049b1ec139 [term 1 FOLLOWER]: Refusing update from remote peer 60c980e4dfdd4342a546d2796bd268c8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:15:02.559468 16740 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 60c980e4dfdd4342a546d2796bd268c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "791b9cac4c3445f6afac29049b1ec139" member_type: VOTER last_known_addr { host: "127.25.92.252" port: 39001 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:15:02.564455 16712 sys_catalog.cc:455] T 00000000000000000000000000000000 P 791b9cac4c3445f6afac29049b1ec139 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 60c980e4dfdd4342a546d2796bd268c8. Latest consensus state: current_term: 1 leader_uuid: "60c980e4dfdd4342a546d2796bd268c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60c980e4dfdd4342a546d2796bd268c8" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 35765 } } peers { permanent_uuid: "b654b353a3e94ddfa4f019b14ef9fcef" member_type: VOTER last_known_addr { host: "127.25.92.253" port: 46123 } } peers { permanent_uuid: "791b9cac4c3445f6afac29049b1ec139" member_type: VOTER last_known_addr { host: "127.25.92.252" port: 39001 } } }
I20260805 23:15:02.564601 16712 sys_catalog.cc:458] T 00000000000000000000000000000000 P 791b9cac4c3445f6afac29049b1ec139 [sys.catalog]: This master's current role is: FOLLOWER
I20260805 23:15:02.566260 16585 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b654b353a3e94ddfa4f019b14ef9fcef [term 1 FOLLOWER]: Refusing update from remote peer 60c980e4dfdd4342a546d2796bd268c8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:15:02.566977 16740 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 60c980e4dfdd4342a546d2796bd268c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b654b353a3e94ddfa4f019b14ef9fcef" member_type: VOTER last_known_addr { host: "127.25.92.253" port: 46123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:15:02.571316 16696 sys_catalog.cc:455] T 00000000000000000000000000000000 P b654b353a3e94ddfa4f019b14ef9fcef [sys.catalog]: SysCatalogTable state changed. Reason: New leader 60c980e4dfdd4342a546d2796bd268c8. Latest consensus state: current_term: 1 leader_uuid: "60c980e4dfdd4342a546d2796bd268c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60c980e4dfdd4342a546d2796bd268c8" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 35765 } } peers { permanent_uuid: "b654b353a3e94ddfa4f019b14ef9fcef" member_type: VOTER last_known_addr { host: "127.25.92.253" port: 46123 } } peers { permanent_uuid: "791b9cac4c3445f6afac29049b1ec139" member_type: VOTER last_known_addr { host: "127.25.92.252" port: 39001 } } }
I20260805 23:15:02.571484 16696 sys_catalog.cc:458] T 00000000000000000000000000000000 P b654b353a3e94ddfa4f019b14ef9fcef [sys.catalog]: This master's current role is: FOLLOWER
I20260805 23:15:02.579303 16711 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60c980e4dfdd4342a546d2796bd268c8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "60c980e4dfdd4342a546d2796bd268c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60c980e4dfdd4342a546d2796bd268c8" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 35765 } } peers { permanent_uuid: "b654b353a3e94ddfa4f019b14ef9fcef" member_type: VOTER last_known_addr { host: "127.25.92.253" port: 46123 } } peers { permanent_uuid: "791b9cac4c3445f6afac29049b1ec139" member_type: VOTER last_known_addr { host: "127.25.92.252" port: 39001 } } }
I20260805 23:15:02.579484 16711 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60c980e4dfdd4342a546d2796bd268c8 [sys.catalog]: This master's current role is: LEADER
I20260805 23:15:02.579617 16748 mvcc.cc:205] Tried to move back new op lower bound from 7315340093671931904 to 7315340093512740864. Current Snapshot: MvccSnapshot[applied={T|T < 7315340093671931904}]
I20260805 23:15:02.580087 16696 sys_catalog.cc:455] T 00000000000000000000000000000000 P b654b353a3e94ddfa4f019b14ef9fcef [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "60c980e4dfdd4342a546d2796bd268c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60c980e4dfdd4342a546d2796bd268c8" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 35765 } } peers { permanent_uuid: "b654b353a3e94ddfa4f019b14ef9fcef" member_type: VOTER last_known_addr { host: "127.25.92.253" port: 46123 } } peers { permanent_uuid: "791b9cac4c3445f6afac29049b1ec139" member_type: VOTER last_known_addr { host: "127.25.92.252" port: 39001 } } }
I20260805 23:15:02.580247 16696 sys_catalog.cc:458] T 00000000000000000000000000000000 P b654b353a3e94ddfa4f019b14ef9fcef [sys.catalog]: This master's current role is: FOLLOWER
I20260805 23:15:02.584157 16711 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60c980e4dfdd4342a546d2796bd268c8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "60c980e4dfdd4342a546d2796bd268c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60c980e4dfdd4342a546d2796bd268c8" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 35765 } } peers { permanent_uuid: "b654b353a3e94ddfa4f019b14ef9fcef" member_type: VOTER last_known_addr { host: "127.25.92.253" port: 46123 } } peers { permanent_uuid: "791b9cac4c3445f6afac29049b1ec139" member_type: VOTER last_known_addr { host: "127.25.92.252" port: 39001 } } }
I20260805 23:15:02.584313 16711 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60c980e4dfdd4342a546d2796bd268c8 [sys.catalog]: This master's current role is: LEADER
I20260805 23:15:02.586217 16712 sys_catalog.cc:455] T 00000000000000000000000000000000 P 791b9cac4c3445f6afac29049b1ec139 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "60c980e4dfdd4342a546d2796bd268c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60c980e4dfdd4342a546d2796bd268c8" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 35765 } } peers { permanent_uuid: "b654b353a3e94ddfa4f019b14ef9fcef" member_type: VOTER last_known_addr { host: "127.25.92.253" port: 46123 } } peers { permanent_uuid: "791b9cac4c3445f6afac29049b1ec139" member_type: VOTER last_known_addr { host: "127.25.92.252" port: 39001 } } }
I20260805 23:15:02.586349 16712 sys_catalog.cc:458] T 00000000000000000000000000000000 P 791b9cac4c3445f6afac29049b1ec139 [sys.catalog]: This master's current role is: FOLLOWER
I20260805 23:15:02.587376 16751 mvcc.cc:205] Tried to move back new op lower bound from 7315340093671931904 to 7315340093512740864. Current Snapshot: MvccSnapshot[applied={T|T < 7315340093671931904}]
I20260805 23:15:02.587565 16749 mvcc.cc:205] Tried to move back new op lower bound from 7315340093671931904 to 7315340093512740864. Current Snapshot: MvccSnapshot[applied={T|T < 7315340093671931904}]
I20260805 23:15:02.593132 16742 catalog_manager.cc:1347] Generated new cluster ID: fe78a40bdc9b48eda8742371047e5cec
I20260805 23:15:02.593245 16742 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:15:02.617703 16742 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:15:02.619100 16742 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:15:02.642340 16742 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 60c980e4dfdd4342a546d2796bd268c8: Generated new TSK 0
I20260805 23:15:02.643365 16742 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260805 23:15:02.685853 16692 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:15:02.682920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39126 (local address 127.25.92.252:39001)
0805 23:15:02.683056 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:02.683062 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:02.684510 (+ 1448us) server_negotiation.cc:407] Connection header received
0805 23:15:02.684753 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:02.684757 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:02.684813 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:02.684882 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:02.685698 (+ 816us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39126: BlockingRecv error: recv got EOF from 127.0.0.1:39126 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260805 23:15:02.688198 16695 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:15:02.683414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36856 (local address 127.25.92.253:46123)
0805 23:15:02.683545 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:02.683550 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:02.686962 (+ 3412us) server_negotiation.cc:407] Connection header received
0805 23:15:02.687336 (+ 374us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:02.687340 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:02.687397 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:02.687454 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:02.688054 (+ 600us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36856: BlockingRecv error: recv got EOF from 127.0.0.1:36856 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260805 23:15:02.689869 16540 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:15:02.686437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52328 (local address 127.25.92.254:35765)
0805 23:15:02.686569 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:02.686574 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:02.688773 (+ 2199us) server_negotiation.cc:407] Connection header received
0805 23:15:02.688972 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:02.688976 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:02.689036 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:02.689095 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:02.689719 (+ 624us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52328: BlockingRecv error: recv got EOF from 127.0.0.1:52328 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260805 23:15:02.695437 16695 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:15:02.693390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36872 (local address 127.25.92.253:46123)
0805 23:15:02.693590 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:02.693596 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:02.694236 (+ 640us) server_negotiation.cc:407] Connection header received
0805 23:15:02.694501 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:02.694505 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:02.694591 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:02.694669 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:02.695311 (+ 642us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36872: BlockingRecv error: recv got EOF from 127.0.0.1:36872 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260805 23:15:02.701874 16695 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:15:02.699711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36878 (local address 127.25.92.253:46123)
0805 23:15:02.699847 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:02.699853 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:02.700586 (+ 733us) server_negotiation.cc:407] Connection header received
0805 23:15:02.700859 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:02.700863 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:02.700926 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:02.700992 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:02.701733 (+ 741us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36878: BlockingRecv error: recv got EOF from 127.0.0.1:36878 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260805 23:15:02.712316 16695 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:15:02.710081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36888 (local address 127.25.92.253:46123)
0805 23:15:02.710236 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:02.710241 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:02.711022 (+ 781us) server_negotiation.cc:407] Connection header received
0805 23:15:02.711344 (+ 322us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:02.711348 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:02.711414 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:02.711474 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:02.712152 (+ 678us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36888: BlockingRecv error: recv got EOF from 127.0.0.1:36888 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
W20260805 23:15:02.721884 16695 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:15:02.719631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36892 (local address 127.25.92.253:46123)
0805 23:15:02.719824 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:02.719830 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:02.720568 (+ 738us) server_negotiation.cc:407] Connection header received
0805 23:15:02.720854 (+ 286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:02.720859 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:02.720927 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:02.720992 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:02.721742 (+ 750us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36892: BlockingRecv error: recv got EOF from 127.0.0.1:36892 (error 108)
Metrics: {"server-negotiator.queue_time_us":102}
W20260805 23:15:02.735236 16695 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:15:02.733052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36900 (local address 127.25.92.253:46123)
0805 23:15:02.733204 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:02.733210 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:02.733997 (+ 787us) server_negotiation.cc:407] Connection header received
0805 23:15:02.734263 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:02.734268 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:02.734336 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:02.734404 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:02.735092 (+ 688us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36900: BlockingRecv error: recv got EOF from 127.0.0.1:36900 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260805 23:15:02.756233 16695 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:15:02.754301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36916 (local address 127.25.92.253:46123)
0805 23:15:02.754443 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:02.754448 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:02.755082 (+ 634us) server_negotiation.cc:407] Connection header received
0805 23:15:02.755344 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:02.755348 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:02.755410 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:02.755468 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:02.756095 (+ 627us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36916: BlockingRecv error: recv got EOF from 127.0.0.1:36916 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260805 23:15:02.792104 16695 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:15:02.790979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36920 (local address 127.25.92.253:46123)
0805 23:15:02.791238 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:02.791242 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:02.791260 (+ 18us) server_negotiation.cc:407] Connection header received
0805 23:15:02.791403 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:02.791406 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:02.791458 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:02.791554 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:02.791992 (+ 438us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36920: BlockingRecv error: recv got EOF from 127.0.0.1:36920 (error 108)
Metrics: {"server-negotiator.queue_time_us":170}
W20260805 23:15:02.859997 16695 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:15:02.858605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36922 (local address 127.25.92.253:46123)
0805 23:15:02.858781 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:02.858786 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:02.858923 (+ 137us) server_negotiation.cc:407] Connection header received
0805 23:15:02.859151 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:02.859154 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:02.859210 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:02.859325 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:02.859856 (+ 531us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36922: BlockingRecv error: recv got EOF from 127.0.0.1:36922 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
W20260805 23:15:02.991904 16695 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:15:02.990655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36926 (local address 127.25.92.253:46123)
0805 23:15:02.990821 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:02.990830 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:02.990973 (+ 143us) server_negotiation.cc:407] Connection header received
0805 23:15:02.991095 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:02.991097 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:02.991149 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:02.991234 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:02.991762 (+ 528us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36926: BlockingRecv error: recv got EOF from 127.0.0.1:36926 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260805 23:15:03.251638 16695 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:15:03.250424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36928 (local address 127.25.92.253:46123)
0805 23:15:03.250760 (+ 336us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:03.250764 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:03.250780 (+ 16us) server_negotiation.cc:407] Connection header received
0805 23:15:03.250837 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:03.250839 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:03.250887 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:03.250972 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:03.251459 (+ 487us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36928: BlockingRecv error: recv got EOF from 127.0.0.1:36928 (error 108)
Metrics: {"server-negotiator.queue_time_us":244}
I20260805 23:15:03.500310 16707 catalog_manager.cc:1259] Loaded cluster ID: fe78a40bdc9b48eda8742371047e5cec
I20260805 23:15:03.500425 16707 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b654b353a3e94ddfa4f019b14ef9fcef: loading cluster ID for follower catalog manager: success
I20260805 23:15:03.502323 16707 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b654b353a3e94ddfa4f019b14ef9fcef: acquiring CA information for follower catalog manager: success
I20260805 23:15:03.503664 16707 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b654b353a3e94ddfa4f019b14ef9fcef: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260805 23:15:03.528075 16739 catalog_manager.cc:1259] Loaded cluster ID: fe78a40bdc9b48eda8742371047e5cec
I20260805 23:15:03.528173 16739 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 791b9cac4c3445f6afac29049b1ec139: loading cluster ID for follower catalog manager: success
I20260805 23:15:03.530162 16739 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 791b9cac4c3445f6afac29049b1ec139: acquiring CA information for follower catalog manager: success
I20260805 23:15:03.531544 16739 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 791b9cac4c3445f6afac29049b1ec139: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260805 23:15:03.770321 16822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:03.765845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36940 (local address 127.25.92.253:46123)
0805 23:15:03.766076 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:03.766080 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:03.766125 (+ 45us) server_negotiation.cc:407] Connection header received
0805 23:15:03.766304 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:03.766307 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:03.766452 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:03.766564 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 23:15:03.766842 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:03.767374 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:03.768357 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:03.768780 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:03.768892 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:03.769304 (+ 412us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 23:15:03.769857 (+ 553us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 23:15:03.769966 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:03.770119 (+ 153us) server_negotiation.cc:299] Negotiation successful
0805 23:15:03.770183 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":93,"threads_started":1}
I20260805 23:15:03.777858 16828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:03.773864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39130 (local address 127.25.92.252:39001)
0805 23:15:03.774119 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:03.774122 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:03.774137 (+ 15us) server_negotiation.cc:407] Connection header received
0805 23:15:03.774226 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:03.774228 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:03.774373 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:03.774485 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 23:15:03.774750 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:03.775274 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:03.776120 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:03.776342 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:03.776418 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:03.776727 (+ 309us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 23:15:03.777242 (+ 515us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 23:15:03.777356 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:03.777609 (+ 253us) server_negotiation.cc:299] Negotiation successful
0805 23:15:03.777683 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":114,"threads_started":1}
I20260805 23:15:03.785277 16828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:03.781678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39138 (local address 127.25.92.252:39001)
0805 23:15:03.781809 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:03.781812 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:03.781826 (+ 14us) server_negotiation.cc:407] Connection header received
0805 23:15:03.781873 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:03.781876 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:03.781926 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:03.782011 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 23:15:03.782291 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:03.782984 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:03.783987 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:03.784216 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:03.784334 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:03.784785 (+ 451us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 23:15:03.784857 (+ 72us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 23:15:03.784965 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:03.785071 (+ 106us) server_negotiation.cc:299] Negotiation successful
0805 23:15:03.785126 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260805 23:15:03.786669 16822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:03.781360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36944 (local address 127.25.92.253:46123)
0805 23:15:03.781533 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:03.781538 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:03.782122 (+ 584us) server_negotiation.cc:407] Connection header received
0805 23:15:03.782206 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:03.782208 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:03.782281 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:03.782359 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 23:15:03.782716 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:03.783549 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:03.785066 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:03.785413 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:03.785529 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:03.786185 (+ 656us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 23:15:03.786310 (+ 125us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 23:15:03.786407 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:03.786511 (+ 104us) server_negotiation.cc:299] Negotiation successful
0805 23:15:03.786563 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260805 23:15:03.787364 16834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:03.781213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52338 (local address 127.25.92.254:35765)
0805 23:15:03.781935 (+ 722us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:03.781940 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:03.781956 (+ 16us) server_negotiation.cc:407] Connection header received
0805 23:15:03.782086 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:03.782089 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:03.782297 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:03.782450 (+ 153us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 23:15:03.782752 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:03.783549 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:03.785061 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:03.785422 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:03.785551 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:03.786294 (+ 743us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 23:15:03.786691 (+ 397us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 23:15:03.786777 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:03.787147 (+ 370us) server_negotiation.cc:299] Negotiation successful
0805 23:15:03.787212 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":628,"thread_start_us":91,"threads_started":1}
I20260805 23:15:03.791224 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 16467
I20260805 23:15:03.799672 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 16543
I20260805 23:15:03.807104 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 16618
2026-08-05T23:15:03Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3321 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971406251336-25971-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 23:15:03 dist-test-slave-ss4n krb5kdc[16840](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 23:15:03 dist-test-slave-ss4n krb5kdc[16840](info): set up 2 sockets
Aug 05 23:15:03 dist-test-slave-ss4n krb5kdc[16840](info): commencing operation
krb5kdc: starting...
W20260805 23:15:04.942394 25971 mini_kdc.cc:121] Time spent starting KDC: real 1.112s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971406251336-25971-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971406251336-25971-0/krb5kdc/test-user.keytab.
Aug 05 23:15:05 dist-test-slave-ss4n krb5kdc[16840](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971705, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T23:15:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:15:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.92.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.92.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.92.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab.
Entry for principal kudu/127.25.92.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab.
I20260805 23:15:05.082015 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971406251336-25971-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971406251336-25971-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971406251336-25971-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.254:40207
--webserver_interface=127.25.92.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab
--principal=kudu/127.25.92.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:45483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.92.254:40207,127.25.92.253:35687,127.25.92.252:43273
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:15:05.200174 16853 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260805 23:15:05.200654 16853 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:15:05.200711 16853 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:15:05.200763 16853 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:15:05.204972 16853 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:15:05.205047 16853 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:15:05.205067 16853 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:15:05.205084 16853 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:15:05.205102 16853 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:15:05.211400 16853 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:45483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971406251336-25971-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971406251336-25971-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.92.254:40207,127.25.92.253:35687,127.25.92.252:43273
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.254.keytab
--principal=kudu/127.25.92.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.254:40207
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.92.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16853
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971406251336-25971-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:15:05.212579 16853 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:15:05.213654 16853 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:15:05.223209 16861 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:05.223444 16859 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:05.223209 16858 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:05.223707 16853 server_base.cc:1034] running on GCE node
I20260805 23:15:05.224542 16853 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:15:05.225868 16853 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:15:05.227102 16853 hybrid_clock.cc:648] HybridClock initialized: now 1785971705227069 us; error 49 us; skew 500 ppm
Aug 05 23:15:05 dist-test-slave-ss4n krb5kdc[16840](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971705, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:15:05.236616 16853 init.cc:377] Logged in from keytab as kudu/127.25.92.254@KRBTEST.COM (short username kudu)
I20260805 23:15:05.237823 16853 webserver.cc:460] Webserver started at http://127.25.92.254:32931/ using document root <none> and password file <none>
I20260805 23:15:05.238438 16853 fs_manager.cc:359] Metadata directory not provided
I20260805 23:15:05.238515 16853 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:15:05.238713 16853 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:15:05.241206 16853 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971406251336-25971-0/minicluster-data/master-0/data/instance:
uuid: "dc0c2f2777c3431782cc5b5fa38c8a86"
format_stamp: "Formatted at 2026-08-05 23:15:05 on dist-test-slave-ss4n"
server_key: "1a90c76d8f622b149daee63402ecf1c2"
server_key_iv: "22dd7d6938028ff09599b444e86e715b"
server_key_version: "encryptionkey@0"
I20260805 23:15:05.241931 16853 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971406251336-25971-0/minicluster-data/master-0/wal/instance:
uuid: "dc0c2f2777c3431782cc5b5fa38c8a86"
format_stamp: "Formatted at 2026-08-05 23:15:05 on dist-test-slave-ss4n"
server_key: "1a90c76d8f622b149daee63402ecf1c2"
server_key_iv: "22dd7d6938028ff09599b444e86e715b"
server_key_version: "encryptionkey@0"
I20260805 23:15:05.246608 16853 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260805 23:15:05.250111 16869 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:05.251915 16853 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260805 23:15:05.252065 16853 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971406251336-25971-0/minicluster-data/master-0/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971406251336-25971-0/minicluster-data/master-0/wal
uuid: "dc0c2f2777c3431782cc5b5fa38c8a86"
format_stamp: "Formatted at 2026-08-05 23:15:05 on dist-test-slave-ss4n"
server_key: "1a90c76d8f622b149daee63402ecf1c2"
server_key_iv: "22dd7d6938028ff09599b444e86e715b"
server_key_version: "encryptionkey@0"
I20260805 23:15:05.252189 16853 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971406251336-25971-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971406251336-25971-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971406251336-25971-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:15:05.272161 16853 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:15:05.275986 16853 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:15:05.276170 16853 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:15:05.283989 16853 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.254:40207
I20260805 23:15:05.284363 16923 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.254:40207 every 8 connection(s)
I20260805 23:15:05.285111 16853 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971406251336-25971-0/minicluster-data/master-0/data/info.pb
I20260805 23:15:05.288385 16924 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:15:05.291038 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 16853
I20260805 23:15:05.291204 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971406251336-25971-0/minicluster-data/master-0/wal/instance
I20260805 23:15:05.291597 25971 external_mini_cluster.cc:1428] Setting key 30baed47a548013eb784cc1e28c6dbe8
I20260805 23:15:05.293205 16924 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.92.254" port: 40207 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.25.92.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.92.253@KRBTEST.COM" created.
Aug 05 23:15:05 dist-test-slave-ss4n krb5kdc[16840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971705, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.254@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM
I20260805 23:15:05.312677 16928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:05.295995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40954 (local address 127.25.92.254:40207)
0805 23:15:05.296625 (+ 630us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:05.296743 (+ 118us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:05.296886 (+ 143us) server_negotiation.cc:407] Connection header received
0805 23:15:05.298654 (+ 1768us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:05.298659 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:05.298888 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:05.299064 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:05.300318 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:05.301323 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:05.302714 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:05.303120 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:05.303261 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:05.306615 (+ 3354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:05.306622 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:05.306634 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:05.306658 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:05.308669 (+ 2011us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:05.309403 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:05.309409 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:05.309410 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:05.309653 (+ 243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:05.310072 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:05.310074 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:05.310076 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:05.310556 (+ 480us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:05.310718 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:05.311139 (+ 421us) server_negotiation.cc:299] Negotiation successful
0805 23:15:05.311519 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":146,"server-negotiator.queue_time_us":494,"spinlock_wait_cycles":52608,"thread_start_us":192,"threads_started":1}
I20260805 23:15:05.312677 16927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:05.295995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:40207 (local address 127.0.0.1:40954)
0805 23:15:05.296670 (+ 675us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:05.296734 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:05.298253 (+ 1519us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:05.299143 (+ 890us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:05.299150 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:05.299722 (+ 572us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:15:05.300121 (+ 399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:05.300145 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:05.301474 (+ 1329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:05.301542 (+ 68us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:05.302536 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:05.302547 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:05.303307 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:05.303310 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:05.303497 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:05.304344 (+ 847us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:15:05.304366 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:15:05.306408 (+ 2042us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:15:05.308828 (+ 2420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:05.308841 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:05.308856 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:05.309197 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:05.309777 (+ 580us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:05.309781 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:05.309787 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:05.309928 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:05.310682 (+ 754us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:15:05.310690 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:15:05.310851 (+ 161us) client_negotiation.cc:769] Sending connection context
0805 23:15:05.311047 (+ 196us) client_negotiation.cc:240] Negotiation successful
0805 23:15:05.311459 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":187,"threads_started":1}
I20260805 23:15:05.315789 16924 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.92.253" port: 35687 } has no permanent_uuid. Determining permanent_uuid...
I20260805 23:15:05.316905 16927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:05.316349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.253:35687 (local address 127.0.0.1:39814)
0805 23:15:05.316589 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:05.316789 (+ 200us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.253:35687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
W20260805 23:15:05.317227 16873 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.25.92.253:35687: connect: Connection refused (error 111)
I20260805 23:15:05.318392 16927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:05.318111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.253:35687 (local address 127.0.0.1:39820)
0805 23:15:05.318254 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:05.318306 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.253:35687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
W20260805 23:15:05.318574 16924 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.92.253:35687: Network error: Client connection negotiation failed: client connection to 127.25.92.253:35687: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.92.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.253.keytab.
Entry for principal kudu/127.25.92.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.253.keytab.
I20260805 23:15:05.326787 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971406251336-25971-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971406251336-25971-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971406251336-25971-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971406251336-25971-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.253:35687
--webserver_interface=127.25.92.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.253.keytab
--principal=kudu/127.25.92.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:45483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.92.254:40207,127.25.92.253:35687,127.25.92.252:43273
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260805 23:15:05.367894 16924 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.92.253" port: 35687 } attempt: 1
I20260805 23:15:05.368974 16927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:05.368536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.253:35687 (local address 127.0.0.1:39824)
0805 23:15:05.368796 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:05.368876 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.253:35687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137}
I20260805 23:15:05.369967 16927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:05.369694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.253:35687 (local address 127.0.0.1:39832)
0805 23:15:05.369834 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:05.369895 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.253:35687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260805 23:15:05.370170 16924 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.92.253:35687: Network error: Client connection negotiation failed: client connection to 127.25.92.253:35687: connect: Connection refused (error 111)
I20260805 23:15:05.438370 16924 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.92.253" port: 35687 } attempt: 2
I20260805 23:15:05.439463 16927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:05.439094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.253:35687 (local address 127.0.0.1:39834)
0805 23:15:05.439316 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:05.439382 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.253:35687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116}
I20260805 23:15:05.440284 16927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:05.440060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.253:35687 (local address 127.0.0.1:39840)
0805 23:15:05.440171 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:05.440227 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.253:35687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260805 23:15:05.440425 16924 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.92.253:35687: Network error: Client connection negotiation failed: client connection to 127.25.92.253:35687: connect: Connection refused (error 111)
W20260805 23:15:05.454044 16931 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260805 23:15:05.454460 16931 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:15:05.454524 16931 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:15:05.454614 16931 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:15:05.459507 16931 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:15:05.459628 16931 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:15:05.459672 16931 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:15:05.459740 16931 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:15:05.459784 16931 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:15:05.466053 16931 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:45483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971406251336-25971-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971406251336-25971-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.25.92.254:40207,127.25.92.253:35687,127.25.92.252:43273
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.253.keytab
--principal=kudu/127.25.92.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.253:35687
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971406251336-25971-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.25.92.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16931
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971406251336-25971-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:15:05.467352 16931 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:15:05.468485 16931 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:15:05.477936 16937 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:05.477974 16936 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:05.477974 16939 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:05.477954 16931 server_base.cc:1034] running on GCE node
I20260805 23:15:05.478982 16931 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:15:05.480294 16931 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:15:05.481572 16931 hybrid_clock.cc:648] HybridClock initialized: now 1785971705481530 us; error 49 us; skew 500 ppm
Aug 05 23:15:05 dist-test-slave-ss4n krb5kdc[16840](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971705, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:15:05.484800 16931 init.cc:377] Logged in from keytab as kudu/127.25.92.253@KRBTEST.COM (short username kudu)
I20260805 23:15:05.485975 16931 webserver.cc:460] Webserver started at http://127.25.92.253:40377/ using document root <none> and password file <none>
I20260805 23:15:05.486613 16931 fs_manager.cc:359] Metadata directory not provided
I20260805 23:15:05.486744 16931 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:15:05.487030 16931 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:15:05.489600 16931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971406251336-25971-0/minicluster-data/master-1/data/instance:
uuid: "77e5c1a557eb489ea28447595e107d69"
format_stamp: "Formatted at 2026-08-05 23:15:05 on dist-test-slave-ss4n"
server_key: "8166bd570bf1a6f2a57ed9f371ddedb5"
server_key_iv: "9caaa8e6224b3aaf26726eddd3df9895"
server_key_version: "encryptionkey@0"
I20260805 23:15:05.490270 16931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971406251336-25971-0/minicluster-data/master-1/wal/instance:
uuid: "77e5c1a557eb489ea28447595e107d69"
format_stamp: "Formatted at 2026-08-05 23:15:05 on dist-test-slave-ss4n"
server_key: "8166bd570bf1a6f2a57ed9f371ddedb5"
server_key_iv: "9caaa8e6224b3aaf26726eddd3df9895"
server_key_version: "encryptionkey@0"
I20260805 23:15:05.494606 16931 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260805 23:15:05.497743 16946 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:05.499231 16931 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:15:05.499409 16931 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971406251336-25971-0/minicluster-data/master-1/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971406251336-25971-0/minicluster-data/master-1/wal
uuid: "77e5c1a557eb489ea28447595e107d69"
format_stamp: "Formatted at 2026-08-05 23:15:05 on dist-test-slave-ss4n"
server_key: "8166bd570bf1a6f2a57ed9f371ddedb5"
server_key_iv: "9caaa8e6224b3aaf26726eddd3df9895"
server_key_version: "encryptionkey@0"
I20260805 23:15:05.499557 16931 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971406251336-25971-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971406251336-25971-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971406251336-25971-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:15:05.524051 16931 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:15:05.527976 16931 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:15:05.528239 16931 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:15:05.531618 16924 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.92.253" port: 35687 } attempt: 3
I20260805 23:15:05.532619 16927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:05.532250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.253:35687 (local address 127.0.0.1:39844)
0805 23:15:05.532441 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:05.532525 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.253:35687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260805 23:15:05.533537 16927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:05.533235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.253:35687 (local address 127.0.0.1:39858)
0805 23:15:05.533361 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:05.533441 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.253:35687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260805 23:15:05.533720 16924 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.92.253:35687: Network error: Client connection negotiation failed: client connection to 127.25.92.253:35687: connect: Connection refused (error 111)
I20260805 23:15:05.536988 16931 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.253:35687
I20260805 23:15:05.537364 16998 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.253:35687 every 8 connection(s)
I20260805 23:15:05.538151 16931 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971406251336-25971-0/minicluster-data/master-1/data/info.pb
I20260805 23:15:05.541445 16999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:15:05.545961 16999 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.92.254" port: 40207 } has no permanent_uuid. Determining permanent_uuid...
I20260805 23:15:05.546605 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 16931
I20260805 23:15:05.546754 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971406251336-25971-0/minicluster-data/master-1/wal/instance
I20260805 23:15:05.547094 25971 external_mini_cluster.cc:1428] Setting key ab4c977d21db8cd88f54f3d95bf7c79f
WARNING: no policy specified for kudu/127.25.92.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.92.252@KRBTEST.COM" created.
Aug 05 23:15:05 dist-test-slave-ss4n krb5kdc[16840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971705, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.253@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM
I20260805 23:15:05.565793 16928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:05.548494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40970 (local address 127.25.92.254:40207)
0805 23:15:05.548697 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:05.548705 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:05.549850 (+ 1145us) server_negotiation.cc:407] Connection header received
0805 23:15:05.551238 (+ 1388us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:05.551242 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:05.551313 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:05.551436 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:05.553072 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:05.553714 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:05.554942 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:05.555209 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:05.555334 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:05.560244 (+ 4910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:05.560251 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:05.560258 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:05.560299 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:05.562486 (+ 2187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:05.563369 (+ 883us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:05.563372 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:05.563374 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:05.563558 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:05.564123 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:05.564126 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:05.564128 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:05.564386 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:05.564612 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:05.565242 (+ 630us) server_negotiation.cc:299] Negotiation successful
0805 23:15:05.565470 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260805 23:15:05.566463 17002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:05.548943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:40207 (local address 127.0.0.1:40970)
0805 23:15:05.549671 (+ 728us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:05.549727 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:05.550973 (+ 1246us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:05.551752 (+ 779us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:05.551761 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:05.552157 (+ 396us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:15:05.552854 (+ 697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:05.552872 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:05.553860 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:05.553863 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:05.554796 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:05.554806 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:05.555386 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:05.555390 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:05.555598 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:05.556380 (+ 782us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:15:05.556401 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:15:05.559996 (+ 3595us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:15:05.562671 (+ 2675us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:05.562682 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:05.562706 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:05.563182 (+ 476us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:05.563703 (+ 521us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:05.563710 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:05.563715 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:05.563983 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:05.564629 (+ 646us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:15:05.564635 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:15:05.564928 (+ 293us) client_negotiation.cc:769] Sending connection context
0805 23:15:05.565107 (+ 179us) client_negotiation.cc:240] Negotiation successful
0805 23:15:05.565425 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":191,"threads_started":1}
I20260805 23:15:05.568179 16999 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.92.253" port: 35687 } has no permanent_uuid. Determining permanent_uuid...
Aug 05 23:15:05 dist-test-slave-ss4n krb5kdc[16840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971705, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.253@KRBTEST.COM for kudu/127.25.92.253@KRBTEST.COM
I20260805 23:15:05.579440 17002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:05.568907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.253:35687 (local address 127.0.0.1:39866)
0805 23:15:05.569091 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:05.569115 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:05.569195 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:05.569956 (+ 761us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:05.569959 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:05.570188 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:15:05.570352 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:05.570358 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:05.571424 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:05.571431 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:05.572071 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:05.572079 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:05.572725 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:05.572729 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:05.572835 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:05.573388 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:15:05.573411 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:15:05.574972 (+ 1561us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:15:05.577560 (+ 2588us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:05.577567 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:05.577573 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:05.577829 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:05.578196 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:05.578199 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:05.578201 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:05.578294 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:05.578918 (+ 624us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:15:05.578921 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:15:05.579039 (+ 118us) client_negotiation.cc:769] Sending connection context
0805 23:15:05.579154 (+ 115us) client_negotiation.cc:240] Negotiation successful
0805 23:15:05.579295 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97}
I20260805 23:15:05.579583 17004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:05.569079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39866 (local address 127.25.92.253:35687)
0805 23:15:05.569451 (+ 372us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:05.569464 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:05.569578 (+ 114us) server_negotiation.cc:407] Connection header received
0805 23:15:05.569622 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:05.569636 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:05.569812 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:05.569980 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:05.570473 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:05.571304 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:05.572200 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:05.572602 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:05.572726 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:05.575157 (+ 2431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:05.575170 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:05.575185 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:05.575212 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:05.577299 (+ 2087us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:05.577972 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:05.577977 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:05.577979 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:05.578069 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:05.578444 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:05.578449 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:05.578451 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:05.578788 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:05.579002 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:05.579250 (+ 248us) server_negotiation.cc:299] Negotiation successful
0805 23:15:05.579387 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":280,"thread_start_us":167,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.92.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.252.keytab.
Entry for principal kudu/127.25.92.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.252.keytab.
I20260805 23:15:05.581804 16999 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.92.252" port: 43273 } has no permanent_uuid. Determining permanent_uuid...
I20260805 23:15:05.582020 25971 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskd06taO/build/debug/bin/kudu
/tmp/dist-test-taskd06taO/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971406251336-25971-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971406251336-25971-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971406251336-25971-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971406251336-25971-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.252:43273
--webserver_interface=127.25.92.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.252.keytab
--principal=kudu/127.25.92.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.92.212:45483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.92.254:40207,127.25.92.253:35687,127.25.92.252:43273
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971406251336-25971-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971406251336-25971-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971406251336-25971-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260805 23:15:05.582648 17002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:05.582362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.252:43273 (local address 127.0.0.1:59798)
0805 23:15:05.582492 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:05.582570 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.252:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260805 23:15:05.582898 16948 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.25.92.252:43273: connect: Connection refused (error 111)
I20260805 23:15:05.584151 17002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:05.583882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.252:43273 (local address 127.0.0.1:59810)
0805 23:15:05.584002 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:05.584081 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.252:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260805 23:15:05.584329 16999 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.92.252:43273: Network error: Client connection negotiation failed: client connection to 127.25.92.252:43273: connect: Connection refused (error 111)
I20260805 23:15:05.633626 16999 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.92.252" port: 43273 } attempt: 1
I20260805 23:15:05.634670 17002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:05.634299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.252:43273 (local address 127.0.0.1:59816)
0805 23:15:05.634472 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:05.634540 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.252:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260805 23:15:05.635576 17002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:05.635277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.252:43273 (local address 127.0.0.1:59820)
0805 23:15:05.635440 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:05.635511 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.252:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
W20260805 23:15:05.635743 16999 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.92.252:43273: Network error: Client connection negotiation failed: client connection to 127.25.92.252:43273: connect: Connection refused (error 111)
I20260805 23:15:05.676955 16924 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.92.253" port: 35687 } attempt: 4
Aug 05 23:15:05 dist-test-slave-ss4n krb5kdc[16840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971705, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.254@KRBTEST.COM for kudu/127.25.92.253@KRBTEST.COM
I20260805 23:15:05.688540 17004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:05.677707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39876 (local address 127.25.92.253:35687)
0805 23:15:05.677880 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:05.677884 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:05.677996 (+ 112us) server_negotiation.cc:407] Connection header received
0805 23:15:05.678221 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:05.678224 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:05.678276 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:05.678434 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:05.679006 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:05.680052 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:05.681844 (+ 1792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:05.682105 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:05.682205 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:05.684819 (+ 2614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:05.684822 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:05.684825 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:05.684853 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:05.686631 (+ 1778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:05.687197 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:05.687200 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:05.687201 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:05.687290 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:05.687602 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:05.687604 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:05.687605 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:05.687854 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:05.687935 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:05.688258 (+ 323us) server_negotiation.cc:299] Negotiation successful
0805 23:15:05.688367 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260805 23:15:05.688549 16927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:05.677632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.253:35687 (local address 127.0.0.1:39876)
0805 23:15:05.677877 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:05.677899 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:05.678058 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:05.678497 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:05.678500 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:05.678722 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:15:05.678861 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:05.678869 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:05.680207 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:05.680210 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:05.681671 (+ 1461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:05.681694 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:05.682245 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:05.682248 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:05.682365 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:05.682922 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:15:05.682937 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:15:05.684638 (+ 1701us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:15:05.686758 (+ 2120us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:05.686764 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:05.686767 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:05.687066 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:05.687399 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:05.687402 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:05.687404 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:05.687509 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:05.687955 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:15:05.687958 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:15:05.688094 (+ 136us) client_negotiation.cc:769] Sending connection context
0805 23:15:05.688209 (+ 115us) client_negotiation.cc:240] Negotiation successful
0805 23:15:05.688374 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130}
I20260805 23:15:05.689600 16924 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.92.252" port: 43273 } has no permanent_uuid. Determining permanent_uuid...
I20260805 23:15:05.690378 16927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:05.690097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.252:43273 (local address 127.0.0.1:59824)
0805 23:15:05.690237 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:05.690299 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.252:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260805 23:15:05.691170 16927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:05.690838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.252:43273 (local address 127.0.0.1:59840)
0805 23:15:05.691015 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:05.691087 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.252:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
W20260805 23:15:05.691368 16924 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.92.252:43273: Network error: Client connection negotiation failed: client connection to 127.25.92.252:43273: connect: Connection refused (error 111)
I20260805 23:15:05.703945 16999 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.92.252" port: 43273 } attempt: 2
I20260805 23:15:05.704852 17002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:05.704558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.252:43273 (local address 127.0.0.1:59846)
0805 23:15:05.704691 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:05.704753 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.252:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260805 23:15:05.705801 17002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:05.705570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.252:43273 (local address 127.0.0.1:59856)
0805 23:15:05.705676 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:05.705736 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.252:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260805 23:15:05.706045 16999 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.92.252:43273: Network error: Client connection negotiation failed: client connection to 127.25.92.252:43273: connect: Connection refused (error 111)
W20260805 23:15:05.709815 17006 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260805 23:15:05.710134 17006 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:15:05.710199 17006 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:15:05.710314 17006 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:15:05.716441 17006 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:15:05.716622 17006 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:15:05.716666 17006 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:15:05.716706 17006 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:15:05.716739 17006 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:15:05.723479 17006 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.92.212:45483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971406251336-25971-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971406251336-25971-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.25.92.254:40207,127.25.92.253:35687,127.25.92.252:43273
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971406251336-25971-0/krb5kdc/kudu_127.25.92.252.keytab
--principal=kudu/127.25.92.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.92.252:43273
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971406251336-25971-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.25.92.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17006
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971406251336-25971-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:15:05.724771 17006 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:15:05.725981 17006 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:15:05.735016 17011 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:05.735080 17014 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:05.735095 17012 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:05.735675 17006 server_base.cc:1034] running on GCE node
I20260805 23:15:05.736353 17006 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:15:05.737691 17006 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:15:05.738901 17006 hybrid_clock.cc:648] HybridClock initialized: now 1785971705738889 us; error 29 us; skew 500 ppm
Aug 05 23:15:05 dist-test-slave-ss4n krb5kdc[16840](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971705, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:15:05.742195 17006 init.cc:377] Logged in from keytab as kudu/127.25.92.252@KRBTEST.COM (short username kudu)
I20260805 23:15:05.743216 17006 webserver.cc:460] Webserver started at http://127.25.92.252:35711/ using document root <none> and password file <none>
I20260805 23:15:05.743800 17006 fs_manager.cc:359] Metadata directory not provided
I20260805 23:15:05.743866 17006 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:15:05.744063 17006 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:15:05.746456 17006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971406251336-25971-0/minicluster-data/master-2/data/instance:
uuid: "6f1152756636427a92fd52ba8d25f8ce"
format_stamp: "Formatted at 2026-08-05 23:15:05 on dist-test-slave-ss4n"
server_key: "440eaefa7e6df5585d0bf1f9773e419d"
server_key_iv: "4782a7d480b23b625d735a9f71b01519"
server_key_version: "encryptionkey@0"
I20260805 23:15:05.747079 17006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971406251336-25971-0/minicluster-data/master-2/wal/instance:
uuid: "6f1152756636427a92fd52ba8d25f8ce"
format_stamp: "Formatted at 2026-08-05 23:15:05 on dist-test-slave-ss4n"
server_key: "440eaefa7e6df5585d0bf1f9773e419d"
server_key_iv: "4782a7d480b23b625d735a9f71b01519"
server_key_version: "encryptionkey@0"
I20260805 23:15:05.750625 16924 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.92.252" port: 43273 } attempt: 1
I20260805 23:15:05.751478 16927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:05.751201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.252:43273 (local address 127.0.0.1:59872)
0805 23:15:05.751342 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:05.751402 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.252:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260805 23:15:05.751624 17006 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260805 23:15:05.752301 16927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:05.752036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.252:43273 (local address 127.0.0.1:59874)
0805 23:15:05.752191 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:05.752242 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.92.252:43273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
W20260805 23:15:05.752501 16924 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.92.252:43273: Network error: Client connection negotiation failed: client connection to 127.25.92.252:43273: connect: Connection refused (error 111)
I20260805 23:15:05.754662 17021 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:05.756198 17006 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:15:05.756342 17006 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971406251336-25971-0/minicluster-data/master-2/data,/tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971406251336-25971-0/minicluster-data/master-2/wal
uuid: "6f1152756636427a92fd52ba8d25f8ce"
format_stamp: "Formatted at 2026-08-05 23:15:05 on dist-test-slave-ss4n"
server_key: "440eaefa7e6df5585d0bf1f9773e419d"
server_key_iv: "4782a7d480b23b625d735a9f71b01519"
server_key_version: "encryptionkey@0"
I20260805 23:15:05.756479 17006 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971406251336-25971-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971406251336-25971-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971406251336-25971-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:15:05.776633 17006 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:15:05.780531 17006 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:15:05.780735 17006 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:15:05.788213 17006 rpc_server.cc:307] RPC server started. Bound to: 127.25.92.252:43273
I20260805 23:15:05.788648 17074 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.92.252:43273 every 8 connection(s)
I20260805 23:15:05.789324 17006 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971406251336-25971-0/minicluster-data/master-2/data/info.pb
I20260805 23:15:05.790459 25971 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskd06taO/build/debug/bin/kudu as pid 17006
I20260805 23:15:05.790666 25971 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskd06taO/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971406251336-25971-0/minicluster-data/master-2/wal/instance
I20260805 23:15:05.790958 25971 external_mini_cluster.cc:1428] Setting key 6e2484d05447df727721dbd35d146bb7
I20260805 23:15:05.792677 17075 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:15:05.797274 16999 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.92.252" port: 43273 } attempt: 3
I20260805 23:15:05.797478 17075 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.92.254" port: 40207 } has no permanent_uuid. Determining permanent_uuid...
Aug 05 23:15:05 dist-test-slave-ss4n krb5kdc[16840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971705, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.92.252@KRBTEST.COM
Aug 05 23:15:05 dist-test-slave-ss4n krb5kdc[16840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971705, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.253@KRBTEST.COM for kudu/127.25.92.252@KRBTEST.COM
Aug 05 23:15:05 dist-test-slave-ss4n krb5kdc[16840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971705, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.252@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM
I20260805 23:15:05.810191 17078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:05.792389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59886 (local address 127.25.92.252:43273)
0805 23:15:05.792823 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:05.792842 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:05.792883 (+ 41us) server_negotiation.cc:407] Connection header received
0805 23:15:05.793803 (+ 920us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:05.793826 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:05.794252 (+ 426us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:05.794640 (+ 388us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:05.795336 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:05.796866 (+ 1530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:05.798461 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:05.799199 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:05.799461 (+ 262us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:05.803078 (+ 3617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:05.803086 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:05.803101 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:05.803138 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:05.806082 (+ 2944us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:05.806601 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:05.806609 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:05.806616 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:05.806754 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:05.807083 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:05.807086 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:05.807088 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:05.807712 (+ 624us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:05.807949 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:05.808728 (+ 779us) server_negotiation.cc:299] Negotiation successful
0805 23:15:05.809047 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":51,"server-negotiator.queue_time_us":287,"thread_start_us":121,"threads_started":1}
I20260805 23:15:05.811147 17002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:05.797935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.252:43273 (local address 127.0.0.1:59896)
0805 23:15:05.798138 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:05.798153 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:05.798238 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:05.798816 (+ 578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:05.798820 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:05.799020 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:15:05.799128 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:05.799137 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:05.800463 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:05.800468 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:05.801369 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:05.801384 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:05.802091 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:05.802097 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:05.802206 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:05.802886 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:15:05.802907 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:15:05.804336 (+ 1429us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:15:05.808811 (+ 4475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:05.808816 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:05.808818 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:05.809153 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:05.809693 (+ 540us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:05.809697 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:05.809700 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:05.809813 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:05.810400 (+ 587us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:15:05.810405 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:15:05.810571 (+ 166us) client_negotiation.cc:769] Sending connection context
0805 23:15:05.810713 (+ 142us) client_negotiation.cc:240] Negotiation successful
0805 23:15:05.810908 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126}
I20260805 23:15:05.811839 17079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:05.798055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59896 (local address 127.25.92.252:43273)
0805 23:15:05.798544 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:05.798557 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:05.798572 (+ 15us) server_negotiation.cc:407] Connection header received
0805 23:15:05.798610 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:05.798613 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:05.798662 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:05.798773 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:05.799265 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:05.800353 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:05.801544 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:05.801942 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:05.802066 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:05.804476 (+ 2410us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:05.804480 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:05.804483 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:05.804550 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:05.808624 (+ 4074us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:05.809326 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:05.809332 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:05.809334 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:05.809544 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:05.809945 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:05.809951 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:05.809955 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:05.810226 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:05.811350 (+ 1124us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:05.811476 (+ 126us) server_negotiation.cc:299] Negotiation successful
0805 23:15:05.811617 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":11,"server-negotiator.queue_time_us":407,"thread_start_us":196,"threads_started":1}
I20260805 23:15:05.812625 16928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:05.800109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40972 (local address 127.25.92.254:40207)
0805 23:15:05.800257 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:05.800263 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:05.800281 (+ 18us) server_negotiation.cc:407] Connection header received
0805 23:15:05.800334 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:05.800338 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:05.800405 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:05.800520 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:05.801286 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:05.802140 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:05.803666 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:05.804079 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:05.804186 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:05.807480 (+ 3294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:05.807485 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:05.807488 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:05.807528 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:05.809419 (+ 1891us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:05.810081 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:05.810085 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:05.810087 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:05.810190 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:05.810580 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:05.810584 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:05.810586 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:05.810836 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:05.810917 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:05.812327 (+ 1410us) server_negotiation.cc:299] Negotiation successful
0805 23:15:05.812460 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260805 23:15:05.812675 17080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:05.799593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.254:40207 (local address 127.0.0.1:40972)
0805 23:15:05.800006 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:05.800062 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:05.800170 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:05.800578 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:05.800584 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:05.800983 (+ 399us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:15:05.801126 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:05.801134 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:05.802262 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:05.802268 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:05.803462 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:05.803478 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:05.804395 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:05.804399 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:05.804562 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:05.805043 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:15:05.805062 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:15:05.807288 (+ 2226us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:15:05.809610 (+ 2322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:05.809619 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:05.809625 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:05.809939 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:05.810325 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:05.810328 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:05.810330 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:05.810440 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:05.811002 (+ 562us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:15:05.811008 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:15:05.811125 (+ 117us) client_negotiation.cc:769] Sending connection context
0805 23:15:05.812255 (+ 1130us) client_negotiation.cc:240] Negotiation successful
0805 23:15:05.812460 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":139,"threads_started":1}
I20260805 23:15:05.814664 17075 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.92.253" port: 35687 } has no permanent_uuid. Determining permanent_uuid...
I20260805 23:15:05.815804 16999 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77e5c1a557eb489ea28447595e107d69: Bootstrap starting.
I20260805 23:15:05.818959 16999 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 77e5c1a557eb489ea28447595e107d69: Neither blocks nor log segments found. Creating new log.
Aug 05 23:15:05 dist-test-slave-ss4n krb5kdc[16840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971705, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.92.254@KRBTEST.COM
I20260805 23:15:05.819675 16924 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.92.252" port: 43273 } attempt: 2
I20260805 23:15:05.819893 16999 log.cc:824] T 00000000000000000000000000000000 P 77e5c1a557eb489ea28447595e107d69: Log is configured to *not* fsync() on all Append() calls
I20260805 23:15:05.822853 16999 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77e5c1a557eb489ea28447595e107d69: No bootstrap required, opened a new log
I20260805 23:15:05.825426 16928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:05.813681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40982 (local address 127.25.92.254:40207)
0805 23:15:05.813823 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:05.813828 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:05.814119 (+ 291us) server_negotiation.cc:407] Connection header received
0805 23:15:05.814257 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:05.814261 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:05.814314 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:05.814414 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:05.815302 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:05.816110 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:05.816991 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:05.817366 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:05.817535 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:05.820778 (+ 3243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:05.820782 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:05.820784 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:05.820811 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:05.822828 (+ 2017us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:05.823473 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:05.823477 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:05.823479 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:05.823574 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:05.823887 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:05.823892 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:05.823895 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:05.824130 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:05.824342 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:05.825022 (+ 680us) server_negotiation.cc:299] Negotiation successful
0805 23:15:05.825151 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
Aug 05 23:15:05 dist-test-slave-ss4n krb5kdc[16840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971705, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.252@KRBTEST.COM for kudu/127.25.92.253@KRBTEST.COM
Aug 05 23:15:05 dist-test-slave-ss4n krb5kdc[16840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971705, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.254@KRBTEST.COM for kudu/127.25.92.252@KRBTEST.COM
I20260805 23:15:05.828398 16999 raft_consensus.cc:348] T 00000000000000000000000000000000 P 77e5c1a557eb489ea28447595e107d69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc0c2f2777c3431782cc5b5fa38c8a86" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 40207 } } peers { permanent_uuid: "77e5c1a557eb489ea28447595e107d69" member_type: VOTER last_known_addr { host: "127.25.92.253" port: 35687 } } peers { permanent_uuid: "6f1152756636427a92fd52ba8d25f8ce" member_type: VOTER last_known_addr { host: "127.25.92.252" port: 43273 } }
I20260805 23:15:05.828665 16999 raft_consensus.cc:729] T 00000000000000000000000000000000 P 77e5c1a557eb489ea28447595e107d69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77e5c1a557eb489ea28447595e107d69, State: Initialized, Role: FOLLOWER
I20260805 23:15:05.829058 16999 consensus_queue.cc:260] T 00000000000000000000000000000000 P 77e5c1a557eb489ea28447595e107d69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc0c2f2777c3431782cc5b5fa38c8a86" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 40207 } } peers { permanent_uuid: "77e5c1a557eb489ea28447595e107d69" member_type: VOTER last_known_addr { host: "127.25.92.253" port: 35687 } } peers { permanent_uuid: "6f1152756636427a92fd52ba8d25f8ce" member_type: VOTER last_known_addr { host: "127.25.92.252" port: 43273 } }
I20260805 23:15:05.830811 16999 sys_catalog.cc:564] T 00000000000000000000000000000000 P 77e5c1a557eb489ea28447595e107d69 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:15:05.831573 17083 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77e5c1a557eb489ea28447595e107d69 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc0c2f2777c3431782cc5b5fa38c8a86" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 40207 } } peers { permanent_uuid: "77e5c1a557eb489ea28447595e107d69" member_type: VOTER last_known_addr { host: "127.25.92.253" port: 35687 } } peers { permanent_uuid: "6f1152756636427a92fd52ba8d25f8ce" member_type: VOTER last_known_addr { host: "127.25.92.252" port: 43273 } } }
I20260805 23:15:05.831730 17083 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77e5c1a557eb489ea28447595e107d69 [sys.catalog]: This master's current role is: FOLLOWER
I20260805 23:15:05.832046 17080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:05.815223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.253:35687 (local address 127.0.0.1:39880)
0805 23:15:05.820781 (+ 5558us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:05.820801 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:05.820934 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:05.821378 (+ 444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:05.821384 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:05.821692 (+ 308us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:15:05.821857 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:05.821869 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:05.823040 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:05.823044 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:05.824000 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:05.824015 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:05.824751 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:05.824757 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:05.824835 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:05.825434 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:15:05.825453 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:15:05.827537 (+ 2084us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:15:05.830024 (+ 2487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:05.830031 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:05.830034 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:05.830318 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:05.830661 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:05.830664 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:05.830666 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:05.830753 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:05.831442 (+ 689us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:15:05.831446 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:15:05.831547 (+ 101us) client_negotiation.cc:769] Sending connection context
0805 23:15:05.831645 (+ 98us) client_negotiation.cc:240] Negotiation successful
0805 23:15:05.831820 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5474}
I20260805 23:15:05.832708 17004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:05.815515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39880 (local address 127.25.92.253:35687)
0805 23:15:05.815645 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:05.815650 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:05.820867 (+ 5217us) server_negotiation.cc:407] Connection header received
0805 23:15:05.821062 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:05.821069 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:05.821132 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:05.821205 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:05.822020 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:05.822889 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:05.824162 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:05.824490 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:05.824566 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:05.827689 (+ 3123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:05.827694 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:05.827699 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:05.827727 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:05.829891 (+ 2164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:05.830464 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:05.830467 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:05.830469 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:05.830557 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:05.830846 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:05.830849 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:05.830851 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:05.831291 (+ 440us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:05.831390 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:05.832397 (+ 1007us) server_negotiation.cc:299] Negotiation successful
0805 23:15:05.832515 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260805 23:15:05.834183 17079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:05.820266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59906 (local address 127.25.92.252:43273)
0805 23:15:05.820424 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:05.820428 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:05.820453 (+ 25us) server_negotiation.cc:407] Connection header received
0805 23:15:05.820494 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:05.820496 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:05.820535 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:05.820597 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:05.821568 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:05.822846 (+ 1278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:05.824174 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:05.824629 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:05.826329 (+ 1700us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:05.828832 (+ 2503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:05.828836 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:05.828839 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:05.828864 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:05.831061 (+ 2197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:05.831673 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:05.831676 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:05.831678 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:05.831913 (+ 235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:05.832301 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:05.832305 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:05.832306 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:05.832495 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:05.832612 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:05.833863 (+ 1251us) server_negotiation.cc:299] Negotiation successful
0805 23:15:05.834021 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260805 23:15:05.834415 17075 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.92.252" port: 43273 } has no permanent_uuid. Determining permanent_uuid...
Aug 05 23:15:05 dist-test-slave-ss4n krb5kdc[16840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971705, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.92.253@KRBTEST.COM
W20260805 23:15:05.836611 17094 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 77e5c1a557eb489ea28447595e107d69: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260805 23:15:05.836683 17094 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Aug 05 23:15:05 dist-test-slave-ss4n krb5kdc[16840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971705, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.92.252@KRBTEST.COM for kudu/127.25.92.252@KRBTEST.COM
I20260805 23:15:05.841225 17082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:05.827036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39884 (local address 127.25.92.253:35687)
0805 23:15:05.827816 (+ 780us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:05.827826 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:05.827838 (+ 12us) server_negotiation.cc:407] Connection header received
0805 23:15:05.827873 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:05.827878 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:05.827937 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:05.828029 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:05.829023 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:05.829950 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:05.831058 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:05.833213 (+ 2155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:05.833317 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:05.836457 (+ 3140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:05.836461 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:05.836464 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:05.836491 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:05.838806 (+ 2315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:05.839340 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:05.839343 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:05.839345 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:05.839454 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:05.839806 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:05.839809 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:05.839811 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:05.840038 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:05.840235 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:05.840900 (+ 665us) server_negotiation.cc:299] Negotiation successful
0805 23:15:05.841029 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":697,"thread_start_us":146,"threads_started":1}
I20260805 23:15:05.842264 16927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:05.820166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.252:43273 (local address 127.0.0.1:59906)
0805 23:15:05.820302 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:05.820321 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:05.820412 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:05.820654 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:05.820657 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:05.820871 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:15:05.820994 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:05.821003 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:05.823002 (+ 1999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:05.823006 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:05.824000 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:05.824013 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:05.824791 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:05.824795 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:05.824874 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:05.825787 (+ 913us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:15:05.825802 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:15:05.828514 (+ 2712us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:15:05.831218 (+ 2704us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:05.831225 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:05.831229 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:05.831513 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:05.832010 (+ 497us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:05.832013 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:05.832015 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:05.832154 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:05.832607 (+ 453us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:15:05.832611 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:15:05.832721 (+ 110us) client_negotiation.cc:769] Sending connection context
0805 23:15:05.833095 (+ 374us) client_negotiation.cc:240] Negotiation successful
0805 23:15:05.833313 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260805 23:15:05.838115 16931 master_runner.cc:186] Error getting master registration for 127.25.92.254:40207: OK, instance_id { permanent_uuid: "dc0c2f2777c3431782cc5b5fa38c8a86" instance_seqno: 1785971705281648 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260805 23:15:05.842517 16931 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260805 23:15:05.845373 25971 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260805 23:15:05.845790 17080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:05.834919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.92.252:43273 (local address 127.0.0.1:59922)
0805 23:15:05.835713 (+ 794us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:05.835730 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:05.835809 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:05.836390 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:05.836393 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:05.836609 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:15:05.836708 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:05.836716 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:05.837692 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:05.837695 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:05.838489 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:05.838514 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:05.839179 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:05.839182 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:05.839259 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:05.839703 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:15:05.839718 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:15:05.841335 (+ 1617us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:15:05.844132 (+ 2797us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:05.844137 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:05.844141 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:05.844386 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:05.844668 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:05.844671 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:05.844672 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:05.844743 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:05.845158 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:15:05.845161 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:15:05.845252 (+ 91us) client_negotiation.cc:769] Sending connection context
0805 23:15:05.845366 (+ 114us) client_negotiation.cc:240] Negotiation successful
0805 23:15:05.845582 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":711}
I20260805 23:15:05.846282 17079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:05.836034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59922 (local address 127.25.92.252:43273)
0805 23:15:05.836130 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:05.836134 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:05.836145 (+ 11us) server_negotiation.cc:407] Connection header received
0805 23:15:05.836175 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:05.836178 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:05.836220 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:05.836288 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:05.836830 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:05.837573 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:05.838632 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:05.838946 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:05.839063 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:05.841521 (+ 2458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:05.841525 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:05.841528 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:05.841553 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:05.844014 (+ 2461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:05.844488 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:05.844491 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:05.844493 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:05.844578 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:05.844831 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:05.844834 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:05.844835 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:05.845037 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:05.845125 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:05.846028 (+ 903us) server_negotiation.cc:299] Negotiation successful
0805 23:15:05.846139 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260805 23:15:05.846443 16924 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc0c2f2777c3431782cc5b5fa38c8a86: Bootstrap starting.
I20260805 23:15:05.846836 16896 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "dc0c2f2777c3431782cc5b5fa38c8a86"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40982
I20260805 23:15:05.849002 16924 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dc0c2f2777c3431782cc5b5fa38c8a86: Neither blocks nor log segments found. Creating new log.
I20260805 23:15:05.849570 17075 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f1152756636427a92fd52ba8d25f8ce: Bootstrap starting.
I20260805 23:15:05.849931 16924 log.cc:824] T 00000000000000000000000000000000 P dc0c2f2777c3431782cc5b5fa38c8a86: Log is configured to *not* fsync() on all Append() calls
I20260805 23:15:05.852290 16924 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc0c2f2777c3431782cc5b5fa38c8a86: No bootstrap required, opened a new log
I20260805 23:15:05.852789 17075 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6f1152756636427a92fd52ba8d25f8ce: Neither blocks nor log segments found. Creating new log.
I20260805 23:15:05.853914 17075 log.cc:824] T 00000000000000000000000000000000 P 6f1152756636427a92fd52ba8d25f8ce: Log is configured to *not* fsync() on all Append() calls
I20260805 23:15:05.855870 16924 raft_consensus.cc:348] T 00000000000000000000000000000000 P dc0c2f2777c3431782cc5b5fa38c8a86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc0c2f2777c3431782cc5b5fa38c8a86" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 40207 } } peers { permanent_uuid: "77e5c1a557eb489ea28447595e107d69" member_type: VOTER last_known_addr { host: "127.25.92.253" port: 35687 } } peers { permanent_uuid: "6f1152756636427a92fd52ba8d25f8ce" member_type: VOTER last_known_addr { host: "127.25.92.252" port: 43273 } }
I20260805 23:15:05.856142 16924 raft_consensus.cc:729] T 00000000000000000000000000000000 P dc0c2f2777c3431782cc5b5fa38c8a86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc0c2f2777c3431782cc5b5fa38c8a86, State: Initialized, Role: FOLLOWER
I20260805 23:15:05.856498 16924 consensus_queue.cc:260] T 00000000000000000000000000000000 P dc0c2f2777c3431782cc5b5fa38c8a86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc0c2f2777c3431782cc5b5fa38c8a86" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 40207 } } peers { permanent_uuid: "77e5c1a557eb489ea28447595e107d69" member_type: VOTER last_known_addr { host: "127.25.92.253" port: 35687 } } peers { permanent_uuid: "6f1152756636427a92fd52ba8d25f8ce" member_type: VOTER last_known_addr { host: "127.25.92.252" port: 43273 } }
I20260805 23:15:05.856647 17075 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f1152756636427a92fd52ba8d25f8ce: No bootstrap required, opened a new log
I20260805 23:15:05.857996 16924 sys_catalog.cc:564] T 00000000000000000000000000000000 P dc0c2f2777c3431782cc5b5fa38c8a86 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:15:05.858035 16896 raft_consensus.cc:482] T 00000000000000000000000000000000 P dc0c2f2777c3431782cc5b5fa38c8a86 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260805 23:15:05.858142 16896 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dc0c2f2777c3431782cc5b5fa38c8a86 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:15:05.859861 16896 raft_consensus.cc:504] T 00000000000000000000000000000000 P dc0c2f2777c3431782cc5b5fa38c8a86 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc0c2f2777c3431782cc5b5fa38c8a86" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 40207 } } peers { permanent_uuid: "77e5c1a557eb489ea28447595e107d69" member_type: VOTER last_known_addr { host: "127.25.92.253" port: 35687 } } peers { permanent_uuid: "6f1152756636427a92fd52ba8d25f8ce" member_type: VOTER last_known_addr { host: "127.25.92.252" port: 43273 } }
I20260805 23:15:05.860922 17075 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6f1152756636427a92fd52ba8d25f8ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc0c2f2777c3431782cc5b5fa38c8a86" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 40207 } } peers { permanent_uuid: "77e5c1a557eb489ea28447595e107d69" member_type: VOTER last_known_addr { host: "127.25.92.253" port: 35687 } } peers { permanent_uuid: "6f1152756636427a92fd52ba8d25f8ce" member_type: VOTER last_known_addr { host: "127.25.92.252" port: 43273 } }
I20260805 23:15:05.861203 17075 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6f1152756636427a92fd52ba8d25f8ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f1152756636427a92fd52ba8d25f8ce, State: Initialized, Role: FOLLOWER
I20260805 23:15:05.861608 16896 leader_election.cc:288] T 00000000000000000000000000000000 P dc0c2f2777c3431782cc5b5fa38c8a86 [CANDIDATE]: Term 1 election: Requested vote from peers 77e5c1a557eb489ea28447595e107d69 (127.25.92.253:35687), 6f1152756636427a92fd52ba8d25f8ce (127.25.92.252:43273)
I20260805 23:15:05.861748 17075 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6f1152756636427a92fd52ba8d25f8ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc0c2f2777c3431782cc5b5fa38c8a86" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 40207 } } peers { permanent_uuid: "77e5c1a557eb489ea28447595e107d69" member_type: VOTER last_known_addr { host: "127.25.92.253" port: 35687 } } peers { permanent_uuid: "6f1152756636427a92fd52ba8d25f8ce" member_type: VOTER last_known_addr { host: "127.25.92.252" port: 43273 } }
I20260805 23:15:05.861791 16973 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "dc0c2f2777c3431782cc5b5fa38c8a86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "77e5c1a557eb489ea28447595e107d69"
I20260805 23:15:05.862100 16973 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 77e5c1a557eb489ea28447595e107d69 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:15:05.862478 17110 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f1152756636427a92fd52ba8d25f8ce [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc0c2f2777c3431782cc5b5fa38c8a86" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 40207 } } peers { permanent_uuid: "77e5c1a557eb489ea28447595e107d69" member_type: VOTER last_known_addr { host: "127.25.92.253" port: 35687 } } peers { permanent_uuid: "6f1152756636427a92fd52ba8d25f8ce" member_type: VOTER last_known_addr { host: "127.25.92.252" port: 43273 } } }
I20260805 23:15:05.862926 17110 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f1152756636427a92fd52ba8d25f8ce [sys.catalog]: This master's current role is: FOLLOWER
I20260805 23:15:05.863359 17075 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6f1152756636427a92fd52ba8d25f8ce [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:15:05.863870 16973 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 77e5c1a557eb489ea28447595e107d69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc0c2f2777c3431782cc5b5fa38c8a86 in term 1.
I20260805 23:15:05.864503 16873 leader_election.cc:302] T 00000000000000000000000000000000 P dc0c2f2777c3431782cc5b5fa38c8a86 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 77e5c1a557eb489ea28447595e107d69, dc0c2f2777c3431782cc5b5fa38c8a86; no voters:
I20260805 23:15:05.865334 17116 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc0c2f2777c3431782cc5b5fa38c8a86 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc0c2f2777c3431782cc5b5fa38c8a86" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 40207 } } peers { permanent_uuid: "77e5c1a557eb489ea28447595e107d69" member_type: VOTER last_known_addr { host: "127.25.92.253" port: 35687 } } peers { permanent_uuid: "6f1152756636427a92fd52ba8d25f8ce" member_type: VOTER last_known_addr { host: "127.25.92.252" port: 43273 } } }
I20260805 23:15:05.865617 17116 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc0c2f2777c3431782cc5b5fa38c8a86 [sys.catalog]: This master's current role is: FOLLOWER
I20260805 23:15:05.865787 17116 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dc0c2f2777c3431782cc5b5fa38c8a86 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:15:05.865921 17116 raft_consensus.cc:686] T 00000000000000000000000000000000 P dc0c2f2777c3431782cc5b5fa38c8a86 [term 1 LEADER]: Becoming Leader. State: Replica: dc0c2f2777c3431782cc5b5fa38c8a86, State: Running, Role: LEADER
I20260805 23:15:05.866389 17116 consensus_queue.cc:237] T 00000000000000000000000000000000 P dc0c2f2777c3431782cc5b5fa38c8a86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc0c2f2777c3431782cc5b5fa38c8a86" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 40207 } } peers { permanent_uuid: "77e5c1a557eb489ea28447595e107d69" member_type: VOTER last_known_addr { host: "127.25.92.253" port: 35687 } } peers { permanent_uuid: "6f1152756636427a92fd52ba8d25f8ce" member_type: VOTER last_known_addr { host: "127.25.92.252" port: 43273 } }
I20260805 23:15:05.867624 17049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "dc0c2f2777c3431782cc5b5fa38c8a86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "6f1152756636427a92fd52ba8d25f8ce"
I20260805 23:15:05.867882 17049 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6f1152756636427a92fd52ba8d25f8ce [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:15:05.869421 17049 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6f1152756636427a92fd52ba8d25f8ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc0c2f2777c3431782cc5b5fa38c8a86 in term 1.
W20260805 23:15:05.869776 17122 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6f1152756636427a92fd52ba8d25f8ce: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260805 23:15:05.869855 17122 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260805 23:15:05.877456 17109 catalog_manager.cc:1558] T 00000000000000000000000000000000 P dc0c2f2777c3431782cc5b5fa38c8a86: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260805 23:15:05.877660 17109 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260805 23:15:05.878013 17098 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc0c2f2777c3431782cc5b5fa38c8a86 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dc0c2f2777c3431782cc5b5fa38c8a86. Latest consensus state: current_term: 1 leader_uuid: "dc0c2f2777c3431782cc5b5fa38c8a86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc0c2f2777c3431782cc5b5fa38c8a86" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 40207 } } peers { permanent_uuid: "77e5c1a557eb489ea28447595e107d69" member_type: VOTER last_known_addr { host: "127.25.92.253" port: 35687 } } peers { permanent_uuid: "6f1152756636427a92fd52ba8d25f8ce" member_type: VOTER last_known_addr { host: "127.25.92.252" port: 43273 } } }
I20260805 23:15:05.878269 17098 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc0c2f2777c3431782cc5b5fa38c8a86 [sys.catalog]: This master's current role is: LEADER
I20260805 23:15:05.878662 16853 master_runner.cc:263] No leader master found from master dc0c2f2777c3431782cc5b5fa38c8a86
I20260805 23:15:05.878736 16853 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260805 23:15:05.881974 17131 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:15:05.883050 17131 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:15:05.889578 17049 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6f1152756636427a92fd52ba8d25f8ce [term 1 FOLLOWER]: Refusing update from remote peer dc0c2f2777c3431782cc5b5fa38c8a86: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:15:05.892190 16973 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 77e5c1a557eb489ea28447595e107d69 [term 1 FOLLOWER]: Refusing update from remote peer dc0c2f2777c3431782cc5b5fa38c8a86: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:15:05.893106 17098 consensus_queue.cc:1035] T 00000000000000000000000000000000 P dc0c2f2777c3431782cc5b5fa38c8a86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f1152756636427a92fd52ba8d25f8ce" member_type: VOTER last_known_addr { host: "127.25.92.252" port: 43273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:15:05.898115 17110 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f1152756636427a92fd52ba8d25f8ce [sys.catalog]: SysCatalogTable state changed. Reason: New leader dc0c2f2777c3431782cc5b5fa38c8a86. Latest consensus state: current_term: 1 leader_uuid: "dc0c2f2777c3431782cc5b5fa38c8a86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc0c2f2777c3431782cc5b5fa38c8a86" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 40207 } } peers { permanent_uuid: "77e5c1a557eb489ea28447595e107d69" member_type: VOTER last_known_addr { host: "127.25.92.253" port: 35687 } } peers { permanent_uuid: "6f1152756636427a92fd52ba8d25f8ce" member_type: VOTER last_known_addr { host: "127.25.92.252" port: 43273 } } }
I20260805 23:15:05.898284 17110 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f1152756636427a92fd52ba8d25f8ce [sys.catalog]: This master's current role is: FOLLOWER
I20260805 23:15:05.898597 17116 consensus_queue.cc:1035] T 00000000000000000000000000000000 P dc0c2f2777c3431782cc5b5fa38c8a86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "77e5c1a557eb489ea28447595e107d69" member_type: VOTER last_known_addr { host: "127.25.92.253" port: 35687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:15:05.904515 17083 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77e5c1a557eb489ea28447595e107d69 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dc0c2f2777c3431782cc5b5fa38c8a86. Latest consensus state: current_term: 1 leader_uuid: "dc0c2f2777c3431782cc5b5fa38c8a86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc0c2f2777c3431782cc5b5fa38c8a86" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 40207 } } peers { permanent_uuid: "77e5c1a557eb489ea28447595e107d69" member_type: VOTER last_known_addr { host: "127.25.92.253" port: 35687 } } peers { permanent_uuid: "6f1152756636427a92fd52ba8d25f8ce" member_type: VOTER last_known_addr { host: "127.25.92.252" port: 43273 } } }
I20260805 23:15:05.904677 17083 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77e5c1a557eb489ea28447595e107d69 [sys.catalog]: This master's current role is: FOLLOWER
I20260805 23:15:05.905097 17082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:05.879395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39898 (local address 127.25.92.253:35687)
0805 23:15:05.879530 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:05.879534 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:05.893833 (+ 14299us) server_negotiation.cc:407] Connection header received
0805 23:15:05.894077 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:05.894082 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:05.894141 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:05.894196 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:05.895289 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:05.896055 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:05.897231 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:05.897617 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:05.897699 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:05.900616 (+ 2917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:05.900620 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:05.900622 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:05.900652 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:05.902586 (+ 1934us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:05.903136 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:05.903140 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:05.903142 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:05.903235 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:05.903787 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:05.903792 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:05.903794 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:05.904187 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:05.904326 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:05.904779 (+ 453us) server_negotiation.cc:299] Negotiation successful
0805 23:15:05.904925 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260805 23:15:05.924273 16928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:05.879058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40994 (local address 127.25.92.254:40207)
0805 23:15:05.879203 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:05.879207 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:05.900138 (+ 20931us) server_negotiation.cc:407] Connection header received
0805 23:15:05.900262 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:05.900265 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:05.900325 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:05.900418 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:05.904212 (+ 3794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:05.905036 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:05.911000 (+ 5964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:05.911379 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:05.911464 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:05.914747 (+ 3283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:05.914751 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:05.914754 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:05.914796 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:05.916675 (+ 1879us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:05.921567 (+ 4892us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:05.921572 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:05.921574 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:05.921684 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:05.922460 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:05.922465 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:05.922466 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:05.922741 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:05.922882 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:05.923931 (+ 1049us) server_negotiation.cc:299] Negotiation successful
0805 23:15:05.924092 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260805 23:15:05.926326 17079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:05.880136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59938 (local address 127.25.92.252:43273)
0805 23:15:05.880263 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:05.880268 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:05.900513 (+ 20245us) server_negotiation.cc:407] Connection header received
0805 23:15:05.900563 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:05.900565 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:05.900608 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:05.900689 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:05.906513 (+ 5824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:05.907352 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:05.909671 (+ 2319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:05.910052 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:05.910139 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:05.919016 (+ 8877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:05.919021 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:05.919025 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:05.919059 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:05.920911 (+ 1852us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:05.922021 (+ 1110us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:05.922025 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:05.922027 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:05.922117 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:05.923215 (+ 1098us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:05.923220 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:05.923221 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:05.923446 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:05.923569 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:05.926021 (+ 2452us) server_negotiation.cc:299] Negotiation successful
0805 23:15:05.926158 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260805 23:15:05.932670 17136 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc0c2f2777c3431782cc5b5fa38c8a86 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "dc0c2f2777c3431782cc5b5fa38c8a86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc0c2f2777c3431782cc5b5fa38c8a86" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 40207 } } peers { permanent_uuid: "77e5c1a557eb489ea28447595e107d69" member_type: VOTER last_known_addr { host: "127.25.92.253" port: 35687 } } peers { permanent_uuid: "6f1152756636427a92fd52ba8d25f8ce" member_type: VOTER last_known_addr { host: "127.25.92.252" port: 43273 } } }
I20260805 23:15:05.932842 17136 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc0c2f2777c3431782cc5b5fa38c8a86 [sys.catalog]: This master's current role is: LEADER
I20260805 23:15:05.932951 17135 mvcc.cc:205] Tried to move back new op lower bound from 7315340107312975872 to 7315340107244654592. Current Snapshot: MvccSnapshot[applied={T|T < 7315340107312975872}]
I20260805 23:15:05.933156 17110 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f1152756636427a92fd52ba8d25f8ce [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "dc0c2f2777c3431782cc5b5fa38c8a86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc0c2f2777c3431782cc5b5fa38c8a86" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 40207 } } peers { permanent_uuid: "77e5c1a557eb489ea28447595e107d69" member_type: VOTER last_known_addr { host: "127.25.92.253" port: 35687 } } peers { permanent_uuid: "6f1152756636427a92fd52ba8d25f8ce" member_type: VOTER last_known_addr { host: "127.25.92.252" port: 43273 } } }
I20260805 23:15:05.933290 17110 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f1152756636427a92fd52ba8d25f8ce [sys.catalog]: This master's current role is: FOLLOWER
I20260805 23:15:05.933300 17137 mvcc.cc:205] Tried to move back new op lower bound from 7315340107312975872 to 7315340107244654592. Current Snapshot: MvccSnapshot[applied={T|T < 7315340107312975872}]
I20260805 23:15:05.938675 17098 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc0c2f2777c3431782cc5b5fa38c8a86 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "dc0c2f2777c3431782cc5b5fa38c8a86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc0c2f2777c3431782cc5b5fa38c8a86" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 40207 } } peers { permanent_uuid: "77e5c1a557eb489ea28447595e107d69" member_type: VOTER last_known_addr { host: "127.25.92.253" port: 35687 } } peers { permanent_uuid: "6f1152756636427a92fd52ba8d25f8ce" member_type: VOTER last_known_addr { host: "127.25.92.252" port: 43273 } } }
I20260805 23:15:05.938853 17098 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc0c2f2777c3431782cc5b5fa38c8a86 [sys.catalog]: This master's current role is: LEADER
I20260805 23:15:05.940187 17083 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77e5c1a557eb489ea28447595e107d69 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "dc0c2f2777c3431782cc5b5fa38c8a86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc0c2f2777c3431782cc5b5fa38c8a86" member_type: VOTER last_known_addr { host: "127.25.92.254" port: 40207 } } peers { permanent_uuid: "77e5c1a557eb489ea28447595e107d69" member_type: VOTER last_known_addr { host: "127.25.92.253" port: 35687 } } peers { permanent_uuid: "6f1152756636427a92fd52ba8d25f8ce" member_type: VOTER last_known_addr { host: "127.25.92.252" port: 43273 } } }
I20260805 23:15:05.940357 17083 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77e5c1a557eb489ea28447595e107d69 [sys.catalog]: This master's current role is: FOLLOWER
I20260805 23:15:05.940702 17139 mvcc.cc:205] Tried to move back new op lower bound from 7315340107312975872 to 7315340107244654592. Current Snapshot: MvccSnapshot[applied={T|T < 7315340107312975872}]
I20260805 23:15:05.944808 17131 catalog_manager.cc:1347] Generated new cluster ID: 71a64715799746d588084b548ed55d6b
I20260805 23:15:05.944911 17131 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:15:05.964318 17131 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:15:05.966068 17131 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:15:05.983740 17131 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dc0c2f2777c3431782cc5b5fa38c8a86: Generated new TSK 0
I20260805 23:15:05.984738 17131 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260805 23:15:06.023453 17082 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:15:06.019642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39906 (local address 127.25.92.253:35687)
0805 23:15:06.019794 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:06.019798 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:06.022096 (+ 2298us) server_negotiation.cc:407] Connection header received
0805 23:15:06.022377 (+ 281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:06.022381 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:06.022458 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:06.022521 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:06.023285 (+ 764us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39906: BlockingRecv error: recv got EOF from 127.0.0.1:39906 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260805 23:15:06.025127 16928 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:15:06.020657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41008 (local address 127.25.92.254:40207)
0805 23:15:06.020786 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:06.020791 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:06.024034 (+ 3243us) server_negotiation.cc:407] Connection header received
0805 23:15:06.024255 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:06.024259 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:06.024322 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:06.024380 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:06.024988 (+ 608us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41008: BlockingRecv error: recv got EOF from 127.0.0.1:41008 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260805 23:15:06.026895 17079 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:15:06.021379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59940 (local address 127.25.92.252:43273)
0805 23:15:06.021554 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:06.021559 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:06.025689 (+ 4130us) server_negotiation.cc:407] Connection header received
0805 23:15:06.026003 (+ 314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:06.026007 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:06.026068 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:06.026127 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:06.026765 (+ 638us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59940: BlockingRecv error: recv got EOF from 127.0.0.1:59940 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260805 23:15:06.032407 17082 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:15:06.030302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39910 (local address 127.25.92.253:35687)
0805 23:15:06.030435 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:06.030440 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:06.031041 (+ 601us) server_negotiation.cc:407] Connection header received
0805 23:15:06.031473 (+ 432us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:06.031478 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:06.031603 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:06.031667 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:06.032281 (+ 614us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39910: BlockingRecv error: recv got EOF from 127.0.0.1:39910 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260805 23:15:06.037036 17082 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:15:06.036108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39926 (local address 127.25.92.253:35687)
0805 23:15:06.036235 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:06.036240 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:06.036256 (+ 16us) server_negotiation.cc:407] Connection header received
0805 23:15:06.036310 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:06.036314 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:06.036368 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:06.036466 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:06.036921 (+ 455us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39926: BlockingRecv error: recv got EOF from 127.0.0.1:39926 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260805 23:15:06.042856 17082 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:15:06.040889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39934 (local address 127.25.92.253:35687)
0805 23:15:06.041009 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:06.041014 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:06.041749 (+ 735us) server_negotiation.cc:407] Connection header received
0805 23:15:06.041957 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:06.041961 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:06.042022 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:06.042080 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:06.042727 (+ 647us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39934: BlockingRecv error: recv got EOF from 127.0.0.1:39934 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260805 23:15:06.051226 17082 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:15:06.049312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39946 (local address 127.25.92.253:35687)
0805 23:15:06.049520 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:06.049525 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:06.050161 (+ 636us) server_negotiation.cc:407] Connection header received
0805 23:15:06.050400 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:06.050403 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:06.050466 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:06.050525 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:06.051085 (+ 560us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39946: BlockingRecv error: recv got EOF from 127.0.0.1:39946 (error 108)
Metrics: {"server-negotiator.queue_time_us":97}
W20260805 23:15:06.063545 17082 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:15:06.061983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39960 (local address 127.25.92.253:35687)
0805 23:15:06.062140 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:06.062145 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:06.062434 (+ 289us) server_negotiation.cc:407] Connection header received
0805 23:15:06.062635 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:06.062639 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:06.062696 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:06.062750 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:06.063410 (+ 660us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39960: BlockingRecv error: recv got EOF from 127.0.0.1:39960 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
W20260805 23:15:06.083794 17082 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:15:06.082455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39966 (local address 127.25.92.253:35687)
0805 23:15:06.082595 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:06.082600 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:06.082860 (+ 260us) server_negotiation.cc:407] Connection header received
0805 23:15:06.083032 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:06.083041 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:06.083093 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:06.083185 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:06.083668 (+ 483us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39966: BlockingRecv error: recv got EOF from 127.0.0.1:39966 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260805 23:15:06.119364 17082 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:15:06.118087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39974 (local address 127.25.92.253:35687)
0805 23:15:06.118235 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:06.118240 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:06.118362 (+ 122us) server_negotiation.cc:407] Connection header received
0805 23:15:06.118578 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:06.118581 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:06.118640 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:06.118738 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:06.119232 (+ 494us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39974: BlockingRecv error: recv got EOF from 127.0.0.1:39974 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260805 23:15:06.187073 17082 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:15:06.185777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39986 (local address 127.25.92.253:35687)
0805 23:15:06.185944 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:06.185948 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:06.186104 (+ 156us) server_negotiation.cc:407] Connection header received
0805 23:15:06.186176 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:06.186179 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:06.186240 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:06.186337 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:06.186915 (+ 578us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39986: BlockingRecv error: recv got EOF from 127.0.0.1:39986 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
W20260805 23:15:06.318894 17082 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:15:06.317614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39990 (local address 127.25.92.253:35687)
0805 23:15:06.317771 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:06.317775 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:06.317916 (+ 141us) server_negotiation.cc:407] Connection header received
0805 23:15:06.318111 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:06.318115 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:06.318169 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:06.318262 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:06.318745 (+ 483us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39990: BlockingRecv error: recv got EOF from 127.0.0.1:39990 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260805 23:15:06.578711 17082 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:15:06.577399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39994 (local address 127.25.92.253:35687)
0805 23:15:06.577623 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:06.577627 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:06.577646 (+ 19us) server_negotiation.cc:407] Connection header received
0805 23:15:06.577816 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:06.577819 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:06.577874 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:06.577957 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:06.578553 (+ 596us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39994: BlockingRecv error: recv got EOF from 127.0.0.1:39994 (error 108)
Metrics: {"server-negotiator.queue_time_us":126}
I20260805 23:15:06.838097 17094 catalog_manager.cc:1259] Loaded cluster ID: 71a64715799746d588084b548ed55d6b
I20260805 23:15:06.838192 17094 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 77e5c1a557eb489ea28447595e107d69: loading cluster ID for follower catalog manager: success
I20260805 23:15:06.839903 17094 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 77e5c1a557eb489ea28447595e107d69: acquiring CA information for follower catalog manager: success
I20260805 23:15:06.841117 17094 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 77e5c1a557eb489ea28447595e107d69: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260805 23:15:06.871270 17122 catalog_manager.cc:1259] Loaded cluster ID: 71a64715799746d588084b548ed55d6b
I20260805 23:15:06.871376 17122 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6f1152756636427a92fd52ba8d25f8ce: loading cluster ID for follower catalog manager: success
I20260805 23:15:06.873028 17122 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6f1152756636427a92fd52ba8d25f8ce: acquiring CA information for follower catalog manager: success
I20260805 23:15:06.874265 17122 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6f1152756636427a92fd52ba8d25f8ce: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260805 23:15:07.097044 17211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:07.092897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39998 (local address 127.25.92.253:35687)
0805 23:15:07.093179 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:07.093183 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:07.093201 (+ 18us) server_negotiation.cc:407] Connection header received
0805 23:15:07.093256 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:07.093258 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:07.093385 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:07.093554 (+ 169us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 23:15:07.093842 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:07.094385 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:07.095275 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:07.095541 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:07.095631 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:07.096043 (+ 412us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 23:15:07.096510 (+ 467us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 23:15:07.096616 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:07.096729 (+ 113us) server_negotiation.cc:299] Negotiation successful
0805 23:15:07.096791 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":87,"threads_started":1}
I20260805 23:15:07.104588 17217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:07.100619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59942 (local address 127.25.92.252:43273)
0805 23:15:07.100866 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:07.100869 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:07.100904 (+ 35us) server_negotiation.cc:407] Connection header received
0805 23:15:07.100984 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:07.100986 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:07.101102 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:07.101214 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 23:15:07.101426 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:07.102062 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:07.102942 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:07.103158 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:07.103277 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:07.103662 (+ 385us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 23:15:07.104139 (+ 477us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 23:15:07.104251 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:07.104351 (+ 100us) server_negotiation.cc:299] Negotiation successful
0805 23:15:07.104403 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":83,"threads_started":1}
I20260805 23:15:07.112025 17217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:07.108159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59956 (local address 127.25.92.252:43273)
0805 23:15:07.108289 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:07.108294 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:07.108700 (+ 406us) server_negotiation.cc:407] Connection header received
0805 23:15:07.108866 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:07.108869 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:07.108912 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:07.108990 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 23:15:07.109224 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:07.109773 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:07.110623 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:07.110861 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:07.110969 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:07.111245 (+ 276us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 23:15:07.111328 (+ 83us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 23:15:07.111421 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:07.111837 (+ 416us) server_negotiation.cc:299] Negotiation successful
0805 23:15:07.111894 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260805 23:15:07.112250 17211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:07.108020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40010 (local address 127.25.92.253:35687)
0805 23:15:07.108149 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:07.108153 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:07.108393 (+ 240us) server_negotiation.cc:407] Connection header received
0805 23:15:07.108553 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:07.108557 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:07.108627 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:07.108685 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 23:15:07.109046 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:07.109807 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:07.111048 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:07.111330 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:07.111455 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:07.111721 (+ 266us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 23:15:07.111816 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 23:15:07.111880 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:07.112034 (+ 154us) server_negotiation.cc:299] Negotiation successful
0805 23:15:07.112095 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260805 23:15:07.112663 17225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:07.107776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41012 (local address 127.25.92.254:40207)
0805 23:15:07.108033 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:07.108039 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:07.108057 (+ 18us) server_negotiation.cc:407] Connection header received
0805 23:15:07.108171 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:07.108175 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:07.108355 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:07.108473 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 23:15:07.108751 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:07.109647 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:07.111041 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:07.111344 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:07.111461 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:07.111847 (+ 386us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 23:15:07.112225 (+ 378us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 23:15:07.112318 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:07.112454 (+ 136us) server_negotiation.cc:299] Negotiation successful
0805 23:15:07.112513 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":91,"threads_started":1}
I20260805 23:15:07.115826 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 16853
I20260805 23:15:07.123437 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 16931
I20260805 23:15:07.130005 25971 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskd06taO/build/debug/bin/kudu with pid 17006
2026-08-05T23:15:07Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3321 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6643 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (300846 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST